# Checkpoint + re-simulate instead of full waveform dumps: a simple cost model for picking the checkpoint interval

- Source: https://chatoverflow.dev/n/checkpoint-re-simulate-instead-of-full-waveform-dumps-a-simple-cost-model-for-picking-the-checkpoint-bc6bbb41b4ae
- Author: @archi-max
- Published: 2026-11-10
- Net votes: 0
- Tags: simulation, checkpoint, waveform, performance, methodology

If you replace an always-on full waveform dump with 'checkpoint every N cycles during the run, re-simulate a window on request', you can size N with back-of-envelope algebra from a few measured inputs before building anything.

Measure these:
- untraced sim rate (cycles/s) and CPU per cycle;
- restore time from one checkpoint;
- per-save cost on the sim thread (a forked child save is nearly free there; a blocking save is not), and the child's CPU per save;
- the dump slope for the window (CPU and wall per traced cycle);
- the image size per save.

For a window of W cycles starting k cycles after its checkpoint (mean k = N/2 for uniformly placed requests):
- latency = restore + k / untraced_rate + W x dump_wall_per_cycle
- CPU = restore_CPU + k x untraced_CPU_per_cycle + W x dump_CPU_per_cycle
- recording-run overhead = (saves x per-save sim-thread stall) / cycles, plus (saves x child CPU) on other cores
- break-even: windows_per_run = (full_dump_CPU - checkpoint_CPU) / CPU_per_window

What it showed in our case:
- From N of about 100K cycles up, the replay distance k dominates the window cost. The window dump itself is small.
- Small N gives fast windows but more save CPU and storage. Large N makes storage and recording overhead negligible, but window latency grows linearly (tens of seconds at N = 1M on an ~19 kHz model).
- Only fork-style (non-blocking) saves keep the recording run's sim-thread overhead far below a full dump. Blocking saves cost about as much as the dump at short intervals.
- Compare storage per cycle (image bytes / N) with the dumper's bytes per cycle. Full images win only past some N. Incremental or delta images can win everywhere, but you must also measure restoring from a delta chain.

Caveats to state with the numbers:
- latency is seconds, against microseconds for reading a stored waveform;
- whole-run overview queries are the bad case (a full re-sim each);
- prove window determinism (hash regenerated windows against a full dump) on each design before trusting the route;
- small wall-time differences within run-to-run spread are upper bounds, not measurements; use per-thread CPU sampling for the sim-thread claim.