Proving a file-format change is backward compatible: build the old reader from its commit and diff its output, don't emulate it

When you add something to a file format and claim 'old readers still work', the cheap check is to emulate the old reader: run the new code with a flag that ignores the new part. That emulation shares the new code's parsing paths and assumptions, so it misses exactly the bugs you're worried about. We had an emulated check passing while real old builds silently degraded: they fell back to a slow path and served some fields as empty instead of failing.

What worked: 1. Pin the oldest reader you promise to support by commit. Build it in a separate worktree: git worktree add /tmp/oldreader <sha>, then build there. Keep it as a test artefact, rebuilt from the SHA in CI, not a binary checked in. 2. Write new-format files with the new writer, then run the OLD binary on them through its normal CLI: open, verify and dump. 3. Assert three things, not just 'it didn't crash': - it takes the same code path as before (for example 'fast open' and not 'fallback scan'), if the tool can report that; - its own verify/integrity command passes; - its full dump is byte-identical to the new reader's dump of the same file. 4. Add the reverse: the new reader on files written by the old writer. 5. Make the old-reader test fail first. Run it against the commit before the compatibility fix and check that it catches the regression.

A cheap design rule that makes this pass: put new data where old readers are already required to skip unknown content (a section type marked optional, an extension field with a length), not in a field old readers interpret.