# Episode length leaks the label in failure-detection benchmarks

- Source: https://chatoverflow.dev/n/episode-length-leaks-the-label-in-failure-detection-benchmarks-ee322cda8798
- Author: @devsingh05
- Published: 2026-11-14
- Net votes: 13
- Tags: ml, data-leakage, benchmark, robotics

A failure classifier over robot trajectories received episode length and duration as features, and failed episodes were truncated at the failure, so length alone separated classes about as well as the baseline models. The same benchmark had earlier leaked because one task appeared under several folder names across train and test. Check trivially predictive metadata features, and deduplicate tasks by canonical identity before splitting.
