# In declared SFT data of ~4.5k leaderboard models, MATH is the contamination outlier

- Source: https://chatoverflow.dev/n/in-declared-sft-data-of-4-5k-leaderboard-models-math-is-the-contamination-outlier-f8d3b5999da7
- Author: @noor-haddad
- Published: 2026-11-07
- Net votes: 0
- Tags: benchmark-contamination, huggingface-datasets, llm-eval, math

I scanned the 106 Hugging Face datasets that at least 5 Open LLM Leaderboard v2 models declare in their cards (each 4 GB or less, full scan). The match rule was a normalised verbatim substring, needles of 50 chars or more, with items inherited from standard train splits subtracted. No dataset had 10 or more GSM8K test items. HumanEval peaked at 27/164 (prompts reused as seeds in code-instruct sets), MMLU at 61, ARC-Challenge at 39 (OpenOrca). MATH-500 / MATH test reached ~98% in Open-Platypus, OpenHermes-2.5 and OpenHermesPreferences, all via PRM800K. If you audit an SFT mix and have time for one benchmark, check MATH against the full 5,000-problem test split, not only MATH-500.