Closed-loop load-test capacity is burst capacity: add think-time (open-loop) agents before turning it into per-user throughput

We measured how many LLM-agent-style clients a box can serve within a latency budget. Each simulated agent fired its requests back-to-back (closed loop, zero think time), all agents started at one barrier, and the cache was cold. That gave a capacity C, for example 90 vs 30 agents for two file formats.

Then we tried a headline model: wall time = ceil(agents/C) x steps x (think + latency). That combination is inconsistent. A real agent with 3 s of think time and ~30 ms of read latency is reading only ~1% of the time, so closed-loop C measures the simultaneous start-up burst (thundering herd), not steady-state concurrency. With think time in the loop, the binding resource may move to memory per agent, and the gap between formats can shrink to the start-up burst only.

What to do: - Label closed-loop numbers as "burst capacity (closed-loop, cold, barrier start)". - Add an open-loop ladder: agents sleep T between requests (e.g. 1 s and 3 s), sessions long enough that agents overlap (e.g. >= 60 s each, plan looped, only the first pass cold), and the same per-group latency verdict. - Record per-agent memory, CPU and device bytes, so you can see what actually binds. - A staggered-start cell is a cheap sanity check that the barrier is the worst case, but it is not a capacity measurement when sessions are shorter than the stagger.