Label stress tests apart from user-facing benchmarks, or a correct number will mislead

Lesson from reporting viewer benchmarks to a non-specialist stakeholder.

We had two kinds of numbers: - **User-facing:** screen recordings of real clicks (open, cursor values, zoom, pan, zoom-to-fit). The viewer only asks for small, screen-sized windows on the visible rows. Here our format was the fastest. - **Stress test:** one request loading every change of 1,000 whole signals (~98 M changes) to compare file formats head-to-head. Here our service came out 3.6x slower than a competitor's reader. The cause was the service repackaging and shipping 100 MB the UI never asks for. The format's own decoder was ~2x faster.

The stress-test table was reported bare, with 'ours: 23.9 s vs theirs: 6.6 s' at the top. The stakeholder read it as 'our product is slower and uses more memory', which is the wrong conclusion for anything users do.

What to do: - Put the workload in the table title: 'stress test: whole-signal load, N signals', not 'loading signals'. - Say up front whether a real user ever triggers this workload, and what the UI actually requests. - Split end-to-end time into stages (decode / encode / transfer) before the headline, so the bottleneck is visible next to it. - For memory, say what the figure includes, e.g. 'RSS summed across 3 processes, including transfer buffers', and that it isn't the format's own footprint. - Keep stress-test results out of anything that looks like the public benchmark page.