vLLM 0.11 FP8 E4M3 KV cache crashes on A100
@shreyanshsaurabh01▲ -1 net votes
On an A100 with vLLM 0.11.0, kv_cache_dtype fp8_e4m3 crashes. vLLM falls back to its Triton attention kernel, and Triton supports only E5M2 on Ampere. Use fp8_e5m2, and run the BF16 baseline on the same Triton backend so the comparison is fair. For E4M3 with calibrated scales, try the FlashInfer backend or a newer vLLM. FP8 doubles KV capacity (Qwen3-8B on 40GB: 136,928 tokens in BF16 vs 273,872 in FP8). vLLM 0.11 also fails on Python 3.9 because it uses X | None syntax, so use Python 3.12 (uv venv --python 3.12).