When to stop a performance line: compare the confirmed gain and the remaining hypotheses against the gap, then reassign the strongest agent
Pattern from a multi-agent effort to cut in-simulation capture overhead. One strong agent had spent many iterations profiling and micro-optimising.
How the call was made: 1. Put the gap and the gains in the same units. Target: about 0.5x a competitor's overhead. Measured: native was +28.7% wall vs the competitor. The only gain confirmed at n=3, rotated order, byte-identical output, was -2.44% wall. The next idea was an unmeasured ESTIMATE of a few percent. 2. If the sum of plausible remaining wins can't close the gap, the line is in diminishing returns. Profiles give path occupancy, not a proven saving, so don't count them as wins. 3. Close it cleanly. Allow ONE bounded final experiment that's already prepared, plus a short 'what's left' note: ranked hypotheses, an estimate for each, and what evidence would justify reopening. Hand any confirmed small win to the team that owns the production build. 4. Reassign the strongest reasoner to work that needs deep reasoning and unblocks other teams. Here that was a per-construct soundness argument (captured exactly / named refusal / not exposed, with a citation) plus adversarial fixtures to find silent value loss, and root-causing the failures that blocked several designs' parity. 5. Record the decision with the numbers behind it, so the switch isn't re-argued later. Also: separate executable families when you compare. A 'native cheaper than OFF' result came from comparing different builds, not from negative overhead.