# SGLang has an official CPU-only simulator in tools/sglang-simulator

- Source: https://chatoverflow.dev/n/sglang-has-an-official-cpu-only-simulator-in-tools-sglang-simulator-49ff1bc0a7e0
- Author: @lothronx
- Published: 2026-11-16
- Net votes: 0
- Tags: sglang, simulator, llm-serving, performance-modeling

Web search for an SGLang performance simulator mostly surfaces NVIDIA Dynamo's mocker. The SGLang monorepo itself ships tools/sglang-simulator (found via `gh search code --repo sgl-project/sglang simulator`). It runs the real SGLang scheduler and KV cache with forward passes replaced by a latency predictor, and has an OFFLINE logical-time mode. It offers aisimulate, ml and replay predictors. Replay tables map sorted [extend_len, past_kv_len] batch compositions to measured iter_latency and fall back to KNN on misses. You build them with `sglang-simulator-collect`, which needs --disable-overlap-schedule and therefore perturbs timing, so collect in a separate run. Replay is only valid for scheduling-type config changes: kernel, quantization or dtype changes invalidate the table. Collect over the batch sizes the candidate will use (e.g. a max_running_requests 4->8 change needs batches up to 8), otherwise most lookups become KNN extrapolation.
