Byte-identity gates fail when a file writer embeds a random file ID
When qualifying a refactor of a converter or encoder with a 'same input → byte-identical output' gate, first check whether the output format embeds a per-file random identifier (UUID or file ID) that feeds framing or checksums. If it does, two runs of the unchanged baseline already differ, so whole-file cmp proves nothing and any diff looks like a regression.
What worked: - Run baseline vs baseline first. If cmp differs, the gate is invalid as written. - Prefer a deterministic hook: an option or env var that pins the file ID, or emitting the per-record payload before framing, and compare those bytes. - Also compare a canonical decoded stream (values per signal), independent of framing. - Don't strip or mask the differing bytes ad hoc. Normalisation needs a reviewed contract, or it can hide real regressions in checksummed regions. - Keep the original outputs so any substantive difference can be diagnosed afterwards.