Low-overhead waveform capture: compare per-change hooks, per-K-cycle polling, state snapshots and fork-based replay by simulation-thread cost

Context: you want full waveform visibility from a long RTL simulation, but with less simulation-thread overhead than the simulator's built-in dumper. We did a desk study using our own measured per-cycle numbers on a ~78K-signal SoC (about 360 value changes per cycle). The built-in dumper added about 14 us/cycle on the simulation thread.

1. Per-change callbacks (DPI/VPI). An empty hook already costs about what the built-in dumper costs in total, so tuning the hook body can at best tie it. 2. One call per K cycles, with change detection compiled into the testbench. The call cost amortises away, but you pay a polling term per monitored signal per cycle (about 13 ns on our fixture). It beats per-change dispatch only below roughly 1-2K monitored signals. Good for small primary sets; it can't scale to all signals. 3. Bulk state memcpy per cycle plus an off-thread diff. A few hundred KB of state per cycle costs 2-4x the dumper's sim-thread cost and produces terabytes to diff. Flops-only plus array write taps is cheaper, but nets must then be re-derived. It needs direct access to the model's state, which is easy with Verilator's model struct and generally unavailable in commercial simulators. 4. Fork-based checkpoint every N cycles: - (a) Lazy replay: keep the snapshots and re-simulate a window when someone opens it. The sim-thread cost is just the fork stall (we measured under 1 us/cycle at N=100K), but windows open in seconds rather than milliseconds. - (b) Eager parallel replay: after each fork, the child turns on the normal dumper and simulates the next N cycles while the parent never dumps. The sim thread pays only the fork stall, every signal ends up recorded in N-cycle chunks, and total CPU is about 2x sim plus dump. Concretely, about 1.7x the dumper's total CPU, plus 3-4 cores per simulation.

Risks for any replay scheme: - non-determinism ($random seeds, time-of-day, external sockets or DPI state); - force/deposit or interactive commands, which aren't replayed; - file descriptors and logs inherited by the forked child; - whether the simulator's dumper behaves correctly in a forked child, which you must test.

The deciding experiment is cheap: one eager-fork arm whose chunk windows are hashed against the same executable's normal dump, plus the parent's sampled sim-thread CPU. Measure the empty-hook floor first. If it is already near the dumper's cost, only options 2 or 4 can win.