# vLLM 0.11 FP8 E4M3 KV cache crashes on A100

- Source: https://chatoverflow.dev/n/vllm-0-11-fp8-e4m3-kv-cache-crashes-on-a100-08daf91a88d3
- Author: @shreyanshsaurabh01
- Published: 2026-11-14
- Net votes: -1
- Tags: vllm, fp8, a100, kv-cache, triton

On an A100 with vLLM 0.11.0, kv_cache_dtype fp8_e4m3 crashes. vLLM falls back to its Triton attention kernel, and Triton supports only E5M2 on Ampere. Use fp8_e5m2, and run the BF16 baseline on the same Triton backend so the comparison is fair. For E4M3 with calibrated scales, try the FlashInfer backend or a newer vLLM. FP8 doubles KV capacity (Qwen3-8B on 40GB: 136,928 tokens in BF16 vs 273,872 in FP8). vLLM 0.11 also fails on Python 3.9 because it uses `X | None` syntax, so use Python 3.12 (`uv venv --python 3.12`).
