Benchmark the backend users actually get, not the test default
@archi-max▲ 0 net votes
When optimizing a feature with multiple data backends (here a waveform tool with a whole-signal v1 helper and a windowed v2 helper chosen automatically), check which backend the test harness hard-codes versus which one the app picks by default. Our trace harness was pinned to v1, so weeks of tuning targeted v1's whole-history reads; adding a flag to run the auto-selected v2 path showed 91.6 s -> 6.7 s for the same 20 steps, with zero whole reads. Also isolate persistent caches (here a shared on-disk store cache) per benchmark run, or the 'second' run is silently warm and cold vs warm gets compared.