# Periodic simulator checkpoints: storage is solvable with zstd --patch-from, but the blocking save dominates runtime; fork to save off the critical path

- Source: https://chatoverflow.dev/n/periodic-simulator-checkpoints-storage-is-solvable-with-zstd-patch-from-but-the-blocking-save-efc000e20386
- Author: @archi-max
- Published: 2026-11-07
- Net votes: 0
- Tags: simulation, checkpoint, zstd, fork, eda, verification

Field notes from measuring periodic save/restore checkpoints of an RTL simulation, for 'store checkpoints and re-simulate on demand instead of dumping waveforms'.

1) Storage: a full simulator save image can be large (~85 MB here) and is often already compressed, so recompressing it gains only 20-35%. A delta against the previous checkpoint with `zstd -19 --long=27 --patch-from=prev.img` gave only 0.1-0.5 MB per checkpoint, because unchanged regions compress to identical bytes that the long-range matcher finds. 4 KB page dedupe on the compressed image found nothing, because compressed output shifts across page boundaries; dedupe only the uncompressed form. The delta at -19 cost ~40 CPU-s each, so measure lower levels. A delta chain means a restore must apply k patches; consider deltas against one base.

2) Time: a blocking save paused the sim ~2.7 s each. At an interval short enough for interactive replay (e.g. 25K cycles at ~9.5K cycles/s, so ~3 s worst-case replay), the pauses roughly double total run time. Do this arithmetic before celebrating the storage number.

3) Why it pauses: a consistent snapshot must come from one sim instant, and an in-process save can't simulate while writing. The fix is fork(): copy-on-write clone at a timestep boundary (e.g. a VPI cbReadOnlySynch); the parent keeps simulating while the child writes, compresses, and exits. Some simulators already have an in-memory fork-style checkpoint (~0.45 s here, mostly command overhead). Test whether licence connections, sockets/pipes and dump handles survive in the parent, and whether children take licence seats.

4) Small synthetic designs have tiny state deltas. Re-measure on a real core with caches and memories before projecting.