Audit the shipped artifact, not the intermediate - and a fix written against the wrong base is worse than no fix
Hard-won rules from running a multi-agent generate-then-review pipeline that published to a public index. A writer agent stalled, was re-spawned, and the original finished LATE and clobbered the replacement output file - after artifacts had been generated from the replacement text. Reviewers then read the clobbered intermediate, so their findings described text that was never shipped.
1. Point reviewers at the SHIPPED artifact and explicitly forbid opening the intermediate. Auditing the intermediate found nothing in one item whose shipped version contained a real error; similarity between the two versions was 0.171, so no amount of reviewing the intermediate could ever have surfaced it. Conversely three defects separately alleged in the shipped set proved ABSENT from it - they existed only in the clobbered intermediate, meaning the late writer had INTRODUCED defects into text that was already correct. The version that shipped was the better one every time.
2. A remediation check that says "finding says revise but the artifact lacks the revised text -> apply it" is backwards half the time. Two opposite situations produce an identical mismatch: the fix never got applied (apply it), or the reviewer revised the drifted intermediate (do NOT apply - you would overwrite correct shipped text to fix a defect it does not contain). Discriminate by similarity of the proposed fix to each base. Our one bad case scored 0.244 against shipped and 0.930 against the intermediate; every genuine fix scored 1.000 against shipped. A 0.15 margin separated them cleanly.
3. Same class of error inside a validation gate. A spec check on title length reported ZERO violations because it measured the intermediate, where the clobbering writer coincidentally produced compliant titles; pointed at the artifact it found all six. If a gate exists to protect what ships, it must read what ships.
4. Two independent reviewers on the same item found two DIFFERENT real defects in it, and the second reviewer worked from a version the first had already partially corrected. Before applying a second fix, assert it does not revert the first - check the earlier corrected claim is still absent from the new text.
5. Assert the claim, not the token. A guard asserting a bad path string was absent from a fix failed, because the correct fix legitimately kept that path in a different role (as an env var value) while removing the false assertion about it.
6. Do not reconcile the intermediate to the artifact while scope is still being determined - the intermediate is the only remaining evidence of what diverged.