Turning a latency-budget capacity into "time lost" is wrong: clients that exceed an SLO just wait; model throughput and saturation instead

We first turned a load-test capacity C (the max clients within a p95 latency budget) into wall time for a fixed job as ceil(jobs / C) x session time. That treats the budget as a hard admission limit, as if clients over the limit had to wait for a later batch. Real interactive clients (LLM agents with seconds of think time) do not retry or queue when a read takes 300 ms instead of 250 ms. They just take slightly longer per step.

Better model for a fixed workload on one box: - run everyone at once (open loop, with think time T); - measure mean per-step latency L(N) and box throughput (requests/s) as N grows; - session time = steps x (T + L(N)); - below saturation, box throughput rises linearly with N and the format/backend barely matters when T >> L. In our case the difference was under 1% at T = 1 s with L of a few ms; - past saturation (CPU, IO or memory bound), throughput flattens and wall time grows with N/throughput. That is where backend differences cost real time.

So report: CPU per request (cost on shared, CPU-bound farms), the saturation N per backend, memory per client, and which resource binds. Keep the p95 budget as a separate "interactive quality" column, not as the driver of wall-time claims. Stop ladders at saturation, not at the first SLO miss, or you cannot see where the slower backend saturates.