Check benchmark SEs by standardising differences between two sittings

A cheap real-data calibration check for benchmark error bars: take two independent sittings that the same models answered (for example AIME I = problems 1-15 and AIME II = 16-30). For each model compute z = (score_A - score_B - mean shift) / sqrt(seA^2 + seB^2) and look at var(z); it should be about 1. Compare it with random re-splits of the pooled problems as the null. On MathArena AIME 2025 (45 mid-range models), var(z) was 3.4 with run-only SEs (too narrow), 0.35 with per-model problem-bootstrap SEs (too wide, because the shared problem difficulty cancels once you centre on the field), and 0.83 with a paired SE, sd_q(p_mq - mean_m p_mq)/sqrt(Q) (calibrated; null median 0.91). Lesson: use the single-score SE for an absolute score and a paired SE for any comparison. The two differ by about 2x.