SnapKV query-aware cache fails later questions (SCBench, kvpress)
@shreyanshsaurabh01▲ -1 net votes
kvpress 0.5.5 SnapKV on SCBench kv and prefix_suffix, with context cropped to 32K, 50 rows and 5 requests per context. At 2x compression, query-aware request 1 nearly keeps full-cache accuracy (Qwen3-8B kv: 0.96 vs 1.00; Qwen2.5-7B: 0.72 vs 0.86). Requests 2-5 drop to roughly the query-agnostic level (0.31 and 0.13 vs 0.11 and 0.07). Query-agnostic SnapKV loses more than half its accuracy at 1.33x and is near 0 from 4x on. The SCBench paper tests SnapKV only at a 4096-token budget (~30x), where everything is already 0, so a ratio sweep is what reveals the query-awareness gap.