Lossless waveform store: code wide packed structs as changed-field XOR rows, not one stream per field

Context: a per-signal block store for simulator waveforms (16K changes per zstd-3 block, XOR-vs-previous or dictionary per block, 4-state as value plane + x/z plane). Wide packed structs (385- and 512-bit trace and flit records with 18-26 fields) were 46% of the stored bytes even after exact-copy dedupe.

Measured on 187 such structs (0.38 G changes), lossless and hash-verified: - whole-struct XOR rows (baseline): 373 MB - one stream per field (each field gets its own times, fed only when it changes): 211 MB (0.57x) - changed-field coding (per change: varint dt + field bitmask + XOR of only the changed fields, laid out column-wise inside each block): 189 MB (0.51x) - best of the three per struct: 175 MB (0.47x). Low-rate structs whose fields change together get BIGGER when split, so choose per signal from a sample block.

Read side (Mac, 6 structs, 2000 px zoomed-in frame, single thread): - baseline: cold 3.1-3.8 ms, random access 0.5 ms - per-field streams: cold 11-12 ms, random access 1.9 ms. Every field's covering block must be decoded, plus a k-way merge to rebuild rows. - changed-field: cold 6.7-7.5 ms, random access 1.0 ms (one block per window). Its encode was also the cheapest: 130-215 ns per struct change vs 213-312 baseline, because it compresses one narrow payload instead of trying XOR and dictionary on 50-64-byte rows. Verdict: use changed-field coding as the struct codec. Use per-field streams only when the viewer shows fields separately and decodes just those.

Two other findings from the same work: 1. An LOD pyramid with 8-cycle buckets cost 45% of the (well-compressed) primary block bytes. Moving the base to 32 cycles dropped it to 5.5%. Measure LOD on typical narrow signals, not only heavy ones: they compress far better than their bucket lists. 2. Per-clock-domain shared timestamps (time header = edge-index deltas or a bitmap over the domain's edge list, plus escapes for off-edge changes) cut time bytes by 42% and total bytes by 12.6% for flop outputs (99% of changes on one clock's posedge). Decode speed didn't change. Random access cost +80 us cold, for one shared 16K-edge chunk.

Implementation gotcha: in an open-addressing dictionary over values, store the 64-bit hash next to each slot and compare it before memcmp. Without that, probing on wide values made the encoder 3x slower, and a profiler showed memcmp as the top frame.