Periodic simulator checkpoints instead of a full waveform dump: CPU wins, blocking save loses on wall time; check what the image contains

We tested saving checkpoints every N cycles during a long RTL run, then making a waveform for any window on demand (restore, re-simulate to the window, dump only it), as an alternative to dumping the whole run. Setup: a VCS testbench calling $save, a CPU core design of ~1M cycles, n=2, shared host.

Findings: - Total CPU: checkpointing every 100K-250K cycles added only ~5-10% of what a whole-run FSDB dump added. The dumper's own writer threads use several times the CPU it costs the simulation thread. - Wall time: a blocking testbench $save stalled the simulation ~4-5 s per save, and only ~1.5-2 s of that was CPU (the rest looked like I/O wait). At 100K spacing that matched the full dump's wall overhead, so it was no faster in wall time. Fewer saves, or a save that runs in a forked child, is what makes the idea pay on wall time. - Image size: list the files in each checkpoint image. Ours was ~100 MB of simulator state plus copies of files the testbench had open (trace logs), and those grew with every save. Keeping every image came to 3-7x the waveform file. Trim or compress before you judge storage; zstd cut whole images ~1.5-4x. - Regeneration latency = restore (grows with image size, 2-4 s here) + re-sim at the run's own speed to the window start + dumping the window. Average and worst case follow from the spacing: worst case is about spacing x per-cycle sim time. - Correctness gate: compare each regenerated window against a slice of a straight dump from the same executable, normalised, before trusting any numbers. 27/27 matched for us.

Report sim-thread CPU, total CPU and wall separately. The answer flips depending on which one you care about.