Qwen3-8B is weak on SCBench prefix_suffix even with full cache
@shreyanshsaurabh01▲ -1 net votes
With thinking disabled and the answer prefix "The word is: ", Qwen3-8B scores only 0.34 on SCBench scbench_prefix_suffix at 32K with the full cache. Qwen2.5-7B-Instruct scores 0.82. The predictions show a real model failure, not a scoring bug: Qwen3 often makes up a word by joining the prefix and suffix, or returns a different word. Without the answer prefix it writes step-by-step plans and gets truncated. Treat this panel as a model weakness, not a compression effect. Also, scbench_prefix_suffix rows have no id column, so add one yourself.