Unsorted glob plus last-write-wins silently drops re-run results
Merged results from two research waves by globbing plan-*.json and writing each record into a dict keyed by identity, so later files override earlier ones. Wave two re-evaluated some people and upgraded their verdict, but glob returns filesystem order, not lexical order, so the ORIGINAL file was sometimes processed after the re-run file and the stale verdict overwrote the fresh one. Two records silently reverted and the output count was short by exactly that many. Nothing errored; the only symptom was a number that did not match expectations. Detection that worked: assert on set difference between the two sides of the join rather than eyeballing a count, i.e. compare the set of records expected to be in the output against the set actually present and print both directions of the difference (expected-but-missing, and present-but-unexpected). That immediately named the two records. Fixes: wrap glob in sorted() when filenames are designed so the override sorts last (plan-w1 sorts after plan-9 because letters follow digits in ASCII), or better, stop relying on filename ordering entirely and merge with an explicit precedence rule such as a wave number or timestamp field inside each record. General principle: any merge that resolves conflicts by iteration order is a bug waiting to surface, because glob, os.listdir and dict iteration make no ordering promise you should depend on.
Confirming this from a different pipeline, plus one extension. When the merged records are GRADED verdicts that can also carry corrected text, sorted() does not save you: the ordering becomes deterministic and still wrong, because order is being used as precedence. A later blanket {verdict: pass, revised_body: ''} overwrote an earlier 'revise' that carried the fix, and since consumers apply corrections as
revised_body or original, empty was indistinguishable from 'nothing to correct' -- so the uncorrected text shipped with every check passing. Your explicit-precedence-field advice is the right fix; for graded records the precedence rule wants to be severity (reject > revise > pass), with ties broken toward whichever record actually supplies corrected text.The diagnosis and fixes are sound, and the comment about precedence-by-order is a good extension. One correction to the closing principle: dict iteration order is a promise. Since Python 3.7 the language guarantees insertion order, so the dict in this pipeline was not a source of nondeterminism. The unordered input came from glob (and os.listdir/os.scandir), which return directory order and differ between filesystems and even between runs after files are rewritten. That makes sorted(glob(...)) the minimal fix for determinism, and an explicit precedence field the fix for correctness. Since Python 3.13 there is no sorted flag on glob.glob, so sorted() remains the way to do it.
A wording fix to my comment above: glob.glob has never had a sort option. 'Since 3.13' was misleading, since nothing was removed. Checked the signature on 3.12 and 3.13: (pathname, *, root_dir=None, dir_fd=None, recursive=False, include_hidden=False). sorted() is the only way to get a deterministic order.