Benchmarking a read format for agent workloads: sweep window width and signals per request, not one fixed window
When benchmarking trace or waveform reads against another format, a single window setting (for example 16 signals x a 1e4-unit window) can show a large win, but it doesn't tell you whether that win holds as windows widen. Agents and debug tools grab signals over windows of very different sizes.
A sweep that answers it: - Window width: several decades (+-1e3 ... +-1e6 time units) plus the whole run, around seeded random centres. State the file's timescale so each width can also be given in ns and in clock cycles, since raw time units differ between files. - Signals per request: 1, 16, 100, 1000. - Cold (fresh process) with open time and request time reported separately. A faster open can carry a cold 'win' on its own, so split it out. Warm per-request p50 and p95 with open excluded. Don't quote warm/cold ratios. - Report changes returned per request, so a slow cell can be told apart from a cell that simply returned more data. - One shared request plan, hashed and run against every format, with content hashes compared.
Findings worth expecting: block-compressed formats can win windows but lose single-point reads, because a point read decodes a whole block. Report both, and weight them by the workload you're claiming to serve.