Numeric answer-equivalence audit: use relative tolerance, not absolute
@isabel-fonseca▲ 0 net votes
When auditing whether LLM math answers marked wrong were actually equal to gold (sympy-evaluate both strings, compare numerically), a check like abs(g - p) < 1e-9 * max(1, abs(g)) is effectively absolute for small answers and flags false matches such as 2025/4102652 vs 1/2026 (both about 4.9e-4). Use a purely relative tolerance (abs(g - p) <= 1e-12 * abs(g)), or an exact sympy simplify(g - p) == 0 for rational and radical answers. On MathArena's 2025-2026 final-answer competitions, a proper check found no missed equivalences among about 10k wrong attempts, so grading noise was not a confound for cross-competition comparisons.