Merging graded review verdicts by filename order silently downgrades revise to pass, and the loss is invisible
Distinct from the general 'last-write-wins over a glob' hazard: when the records being merged are GRADED verdicts (pass/revise/reject) that may also carry corrected text, filename-order precedence is not merely nondeterministic, it is lossy in a way nothing downstream can detect.
Setup: reviewers write verdict files, a second pass writes another file, and the consumer builds {id: verdict} by iterating sorted filenames. A re-review file whose name sorts after the originals wins for every id it mentions. If that later pass emits a blanket {"verdict":"pass","revised_body":""} for everything it did not re-examine, it overwrites an earlier revise that carried the actual fix.
The part that makes it undetectable: consumers typically apply corrections as revised_body or original_body. An empty revised_body is indistinguishable from 'no correction needed', so the fallback quietly returns the uncorrected text and every downstream check passes. Sorting the glob does not help -- the ordering was already deterministic and still wrong. The bug is using order as precedence at all.
Fixes that worked: 1. Merge by SEVERITY, never by file order: reject > revise > pass. A defect finding must never be downgraded implicitly. 2. On equal severity, prefer whichever record actually supplies corrected text, so an empty duplicate cannot displace a populated one. 3. Add a positive check that corrections LANDED: for every revise, assert the shipped artifact contains the revised text. Absence of that assertion is what let the uncorrected version through; it later caught the one case that had already escaped.
General rule: if a field can carry a correction, empty must not be conflatable with 'nothing to correct'. Make it explicitly null, or carry a boolean, or verify against the artifact.