Waveform format conversion: the savings come from dedupe and a tight format, not heavy compression; the slow part is reading the source

Measured while comparing a custom per-signal block format (varint time deltas + XOR/dictionary values + zstd, with a LOD pyramid) against a 1.7 GB FSDB from a SoC simulation. Every signal is stored outright, including the ones FSDB recomputes on read.

Size, whole file (projected from per-instance measurements): - zstd-3 blocks, no dedupe: about 1.06x larger than the FSDB. - Plus storing exact-duplicate streams once: 1.7x smaller. Port copies of the same net were about 42% of bytes. - Plus zstd-19: 2.2x smaller. That's only ~25% more than dedupe alone, for ~13x the encode CPU (1.2-1.4 us/change vs ~0.1 us/change). Decode speed is the same at either level. - The same picture held on a 10 GB dump (0.27 B/change on disk in both): ~6.0 GB vs ~4.6 GB.

Takeaways: 1. Dedupe plus a compact format is the cheap win. Strong entropy coding is 'pay compute for a bit more' and belongs in an idle background recompress, never on the path to first draw. 2. Converting from FSDB is limited by the reader, not the encoder: rebuilding computed signals plus reader overhead came to ~1.3-3.1 us/change per process. A 10 GB file took an estimated 3-8 h with 4 processes. Reader RSS grew with the changes walked (4.3 GB for one 512-bit signal over the full run), so big files need time-sliced, fresh processes. 3. The interesting alternative is writing your format during the simulation. The encoder at ~10M changes/s is in the same range as the FSDB writer (~7M changes/s, measured during the sim), so dump overhead could be similar. VPI callbacks on every signal may be too slow, though; that's what a native dumper hook avoids. 4. Frame rendering cost from such blocks doesn't depend on file size: cold p95 4-13 ms, warm under 1 ms, for 64-100 heavy signals on 8 decode threads.