Compression-level sweeps inside a running simulation: judge on paired clean blocks, and compare size against the competitor's smallest setting, not its live default

Field note from sweeping zstd levels for a waveform writer dumping live from a simulator (7 arms x 3 reps plus a recheck), against FST and another vendor format.

Lessons: - Session drift between rep blocks (about 0.1 s of about 21 s total CPU here) is bigger than many per-arm differences. Compare arms paired within the same block, and treat differences below the drift (about 0.15 s) as 'no difference detected'. Claim a win only if it holds in every clean block. - Measure the post-close fsync barrier separately for every arm. If one writer also fsyncs inside its own process, label that, since it counts against that writer. - Higher levels can cost much more for little gain. zstd-19 cost 2-3.7x the reference format's dump CPU for only about 6% fewer bytes than zstd-9. Set a minimum size gain (e.g. 10%) and a CPU budget relative to the reference before adopting an aggressive level. - Report dump cost as a share of total simulation CPU (here 2.5-4.6% for every format). It tells you whether spending more write CPU for size or read speed is affordable at all. - Compare size against the competitor's smallest achievable setting, even if that's an offline post-process. FST's live LZ4 output can be about twice the size of zlib + repack. A win against the live setting isn't a win against FST at its best. Converting the same dump offline is cheap and needs no quiet slot. - 'Reads' that only touch time, handle and the first value byte aren't full reads. Keep them out of headline read claims. - Report bytes both for the main file and for all files the dump leaves, sidecars included.