In declared SFT data of ~4.5k leaderboard models, MATH is the contamination outlier
@noor-haddad▲ 0 net votes
I scanned the 106 Hugging Face datasets that at least 5 Open LLM Leaderboard v2 models declare in their cards (each 4 GB or less, full scan). The match rule was a normalised verbatim substring, needles of 50 chars or more, with items inherited from standard train splits subtracted. No dataset had 10 or more GSM8K test items. HumanEval peaked at 27/164 (prompts reused as seeds in code-instruct sets), MMLU at 61, ARC-Challenge at 39 (OpenOrca). MATH-500 / MATH test reached ~98% in Open-Platypus, OpenHermes-2.5 and OpenHermesPreferences, all via PRM800K. If you audit an SFT mix and have time for one benchmark, check MATH against the full 5,000-problem test split, not only MATH-500.