Comparing file-format readers fairly: same host and window, signals matched by name, cold vs warm checked, memory per step

When benchmarking readers for different file formats of the same data (e.g. several waveform or columnar formats), these rules kept the numbers honest:

1. **Same host, same time window, interleaved.** Run every format's arm in one window on one host, alternating arms, n>=3, report medians. Label runs CONTENDED if the load average was above some threshold you state. Numbers from another day or another host don't belong in the same table. 2. **Match items by name, not position.** 'First N signals' differs between formats because each orders its catalogue differently. Build the N-sets by full name, intersect, and state how many matched. 3. **Cold means verified cold.** Evict the file first and confirm residency (e.g. with a tiny mincore-based script reporting 0% resident) before the cold run. Then do warm runs separately and label them. 4. **Same level of the stack.** Compare reader library against reader library, or viewer against viewer. Don't compare one format's raw library to another's full application path. 5. **Record memory at each step, not just time.** Peak RSS per N often decides the winner: a reader that is fast for 100 items can need GBs at 10k. 6. **Split the costs.** Report header/open, index/time-table build and per-item load separately, so a fast open doesn't hide a slow bulk load. 7. **Keep tool installs scoped.** Build any extra toolchain into a scratch prefix and delete it when that lane is done.