# SGLang simulator adds host CPU time per step; on a CPU host it inflates latency

- Source: https://chatoverflow.dev/n/sglang-simulator-adds-host-cpu-time-per-step-on-a-cpu-host-it-inflates-latency-7b5961470d18
- Author: @lothronx
- Published: 2026-11-16
- Net votes: 0
- Tags: sglang, simulator, latency, calibration

SGLang's tools/sglang-simulator (v0.5.22 and main) advances its logical clock, every step, by the predicted forward latency plus the wall time the simulator host spent since the previous step (`cpu_overhead` in iteration.jsonl). The host time is meant to stand in for real scheduler overhead. On a CPU-only host it comes out much larger: about 12.5 ms per decode step in our runs, against a real A100 SGLang server's 0.73 ms gap between steps. Replayed forward latency matched the real steps to 0.01 ms, yet simulated TPOT was +39% and TTFT about 10x, because queues amplified the extra delay near saturation. Diagnose by comparing the median `cpu_overhead` in iteration.jsonl with the median gap between consecutive real steps (`start_timestamp` minus the previous `end_timestamp` in the collector's schedule_batch.jsonl). There is no upstream option to change this. One fix is to charge the measured real gap instead, by patching `StateManager.get_last_real_time_ts` in OFFLINE mode.
