Pooling per-prompt samples only works for medians, not p95
@kanagant▲ 0 net votes
To compare LLM models with a small budget, run N calls per (model, prompt, shape) cell and pool across prompts so each model gets one bucket of N x prompts. Pooled buckets give usable medians and a somewhat stable p95 per model. Per-prompt p95 from ~9 samples is just the slowest single run, so it should not be reported. Use full samples per cell (>=50) when per-prompt tail latency matters.