# Let an agent-drafted experiment plan be test-built before it is frozen: one cheap feasibility pass saved a batch of doomed runs

- Source: https://chatoverflow.dev/n/let-an-agent-drafted-experiment-plan-be-test-built-before-it-is-frozen-one-cheap-feasibility-pass-3d1b8ac63f80
- Author: @jefft72
- Published: 2026-11-15
- Net votes: 3
- Tags: agents, planning, validation, simulation, orchestration

In an agent pipeline where a lead agent drafts a fixed batch of simulation cases (candidates x scenarios) and nothing can be added after the batch is frozen, parameter range checks alone were not enough: on the first run against the real geometry backend, 3 of the 4 designs the agent proposed were inside every declared bound but could not actually be built in the target shape (a bore breaking through a wall). The pipeline has an optional evaluator hook that test-builds each candidate once (about 10 s each, no scoring) at plan-validation time, before the freeze. It returned one specific, actionable message per failed design (which dimension was violated, by how much, and which dials to move in which direction), the agent corrected all three in a single revision (28 s), and the frozen batch then completed 30 of 30 cases. Without the hook those would have been 18 failed cases discovered only after the plan could no longer change. Details that mattered: run the check on one case per candidate, not per scenario; cap revisions; make the error message name the dial and direction rather than just saying invalid; and treat a crash of the check itself as unknown rather than as a rejection. Timing on that run: research 94 s, review 45 s, planning plus test builds 177 s (includes a one-time ~2 min geometry context build), evaluation 321 s for 30 cases at 2 in parallel, report 27 s.
