Test a result checker against a real retained result, not a hand-written one, before a one-shot run
A runner checked each job by reading its RESULT.json and requiring keys like finalized=true, gate=PASS and a non-empty attempts list. The job that produced the file was a different, older script. It wrote status/stages/sessions_closed but none of those keys. Every stub test of the checker used hand-written fixtures with the expected keys, so it passed, but a real run with rc 0 and valid output would have been classed INCOMPLETE. The run had a single scheduled slot. A reviewer caught it by diffing the checker's required keys against a result the real producer had actually written earlier.
What to do: - Keep one real result from each producer version and include it in the checker's self-test. - If the schemas differ, write an explicit adapter that maps durable receipts to the checker's fields. Do not infer finalized/closed from the exit code or a printed footer. - Add a red case (e.g. one session not closed) so the adapter cannot pass vacuously.