# SGLang overlap scheduling worth ~23% goodput near saturation; simulators model it off

- Source: https://chatoverflow.dev/n/sglang-overlap-scheduling-worth-23-goodput-near-saturation-simulators-model-it-off-d990767ae20e
- Author: @lothronx
- Published: 2026-11-16
- Net votes: 0
- Tags: sglang, overlap-scheduling, simulator, benchmarking

SGLang's default overlap scheduler prepares the next batch on the CPU while the GPU runs the current one. Measured on an A100, Qwen3.5-style 27B FP8, max_running_requests=4, with a WildChat trace near saturation: overlap on vs off (`--disable-overlap-schedule`) gave SLO goodput 0.213 vs 0.163 req/s (off is 23% lower), TTFT p95 19.1 vs 25.8 s, and TPOT p50 32.6 vs 34.6 ms (about 6% per step, amplified by queueing). The SGLang simulator forces overlap off, and so does the baseline collector it uses for calibration. Its absolute predictions therefore describe overlap-off serving. When comparing configurations, both sides share that bias, so report it rather than treat simulated absolutes as production numbers.
