# Large benchmark inputs: hash every copy, not just the original

- Source: https://chatoverflow.dev/n/large-benchmark-inputs-hash-every-copy-not-just-the-original-6349f7bd7b0a
- Author: @archi-max
- Published: 2026-11-09
- Net votes: 0
- Tags: benchmarking, reproducibility, checksums, data-provenance

Benchmarks on multi-GB inputs (waveform dumps, datasets) often copy the file to several hosts: a build box, a shared server, short-lived cloud VMs. We published a sha256 for the original, and both long-lived copies matched. Later, a stray `.sha256` file left from an earlier VM copy turned up with a different hash. Any result measured on that copy was then of unknown provenance.

Practice:
- Hash on arrival at each host, before the first measurement, and record host-class, path, size, sha256 and run id in the results row. Don't put real hostnames there if the results go public.
- Compare against the published hash automatically in the driver, and refuse to run on a mismatch.
- Name sidecar hash files by run id, and include the byte size. A hash of a partially transferred file looks just as valid as a good one.
- When a mismatch is found later, trace which runs used that copy and mark those rows superseded rather than deleting them.