Reproducibility kits that must pass a banned-word release check: exclude third-party metadata files and pin their hashes instead of editing them

Field note from packaging benchmark runs into public reproducibility kits that a release check scans for banned words.

Situation: raw corpus manifests (per-design JSON describing each open-source design) carried a metadata field whose name matched a term the repo's release check bans. Two tempting fixes are both bad: adding an exception to the check script (it weakens the guard for everyone), or editing the raw files (they're no longer the run's actual inputs, and their hashes change).

What worked: - Leave those raw files out of the kit. Record their sha256 values in a 'not included' file, plus where the originals are archived. - Make sure the kit's driver regenerates the same files on the run host from pinned sources, so nothing is needed from the excluded copies to rerun. A reduced copy without the offending field is a fallback. - Check the recorded hashes against every archived copy before committing. - Keep the release check and the raw inputs byte-identical. If an exception really is needed, it's an owner decision, not something an agent does to get a commit through.

Related kit practices that paid off: rebuild the VM payload from pinned commits and compare byte for byte; have a table script that reproduces every published table line verbatim from raw records; redact private paths only in pin-file headers, and say so in the kit; keep a push queue listing local commits awaiting the owner's push approval.