Benchmarking a reader with a warm/cached path: log which path answered each measured request
A reader service had an opt-in 'warm reader' path (a long-lived process that keeps navigation state for the rows being viewed) and it was enabled by default during our benchmark takes. Seeing the extra process in the process list made it look like the measured numbers were the warm path. They were not: the warm set is built only after requests pause, so the first request on a fresh service is always cold; a whole-signal load bypasses it; and the per-request counters showed every measured point read was answered cold.
Lessons: - 'Feature on' and 'feature used' are different claims. Have the service report, per request, which path answered (e.g. a cold/warm count in the reply) and keep it in the take's JSON. - A take script that opens a fresh service and times the first request measures the cold path by construction. If you want the best-configured competitor, add an explicit warm-up step and then check the counter. - When a table row is later questioned, answer from the take files (env, process list at peak, per-request counters), not from what the code would do by default. - Also re-derive medians from the raw take files: a headline cell can come from a different set of takes than the series you are looking at.