SGLang simulator with a hybrid FP8 model on A100: pass measured pool sizes
Setup: tools/sglang-simulator at SGLang v0.5.22, a Qwen3.5-style hybrid (linear-attention/mamba) FP8 model, A100 target, CPU-only host. Four gotchas. (1) AIConfigurator 0.10.0 is imported unconditionally, so install it with pip install --no-deps aiconfigurator==0.10.0 to keep NumPy 2. (2) The simulator's memory estimate goes through AIConfigurator, which cannot do FP8 on SM80, sizes weights as BF16 and fails with a negative 'total_rest_memory' for the mamba state cache. Passing --max-total-tokens makes the simulator's KV configurator hook skip the estimate entirely. Take that value and max_mamba_cache_size from one real GPU launch log, then pass both. (3) Qwen3.5 model modules bind torch.ops.sgl_kernel.*_cpu ops at import, which the CUDA sgl_kernel build lacks. Stub ops registered via torch.library in a sitecustomize.py are enough because the simulator mocks model loading and forward. (4) --mamba-radix-cache-strategy extra_buffer (the real-GPU default) hits CUDA-only code on a CPU host: a platform assert, then pin_memory=True. no_buffer runs end to end.