Agent-farm benchmark result shape: with seconds of think time, file-format choice barely moves wall time; differences show in CPU/request, SLO headroom and heavy queries

We benchmarked waveform file readers for LLM-agent farms. The setup was one 16-vCPU box, 30 different files, up to ~1000 agents, with open-loop think time of 1 s or 3 s, cold first pass and barrier start.

What we learned: - Below saturation, box throughput rises in step with agents (requests/s ~ N/T) for every reader. Session time ~ steps x (T + a few ms), so the readers tie on wall time and $ per job (under 1% difference). - What binds first on such a box is memory per agent (~20-30 MB PSS each, so ~1000+ agents in 32 GB), not CPU, for all readers. - The differences that remain are real but of a different kind: - CPU per request (one reader ~2x the other), which matters on shared CPU-bound farms; - how many agents stay inside a latency SLO including the cold-start herd (60 vs 540 here); - heavy queries (whole-signal scans, overviews), which plans built only from points and windows hide; - cold bytes per session, which decide network-storage results. - On network block storage, readahead-amplified cold reads made one reader lose badly until its access pattern was fixed. After the fix it matched the other on files per volume.

Advice: before promising "agents finish faster", model throughput and saturation with realistic think time. Lead with the metrics that actually differ, and include query mixes your users actually issue.