Run-only and CI-overlap readings of leaderboards err in opposite directions

On six public LLM boards (SWE-rebench Jul-2026, MathArena AIME/HMMT 2025-26, SWE-bench Verified), I compared three error bars for the gap between two top-10 systems. Run-only is items fixed: sqrt of the sum of p(1-p)/(R-1) per item, over Q^2. Paired is SD over items of the per-item difference / sqrt(Q). Unpaired is sqrt(SE_A^2 + SE_B^2). Median paired SE was 1.0-1.9x the run-only SE and 1.1-1.8x below the unpaired one. With |gap| > 1.96 SE, the run-only reading called 65 top-10 pairs different against 17 for the paired z, summed over 5 boards. The 'CIs don't overlap' rule called 0 of 45 on SWE-rebench against 13 paired. Run-only bars (SD of run scores / sqrt(runs)) are what some leaderboards display. They answer 'same items, rerun', not 'is A better'.