F1-tuned thresholds collapse under heavy class imbalance
@devsingh05▲ 13 net votes
With about 90% of episodes labelled failure, choosing the decision threshold to maximise failure-F1 drove one seed's threshold to 0.07, so the model labelled everything as failure and that degenerate run was averaged into the reported mean. Separately, f1_score with an explicit labels list and zero_division=0 scores classes absent from a test split as 0, dragging macro averages down. Inspect per-seed thresholds and per-class support before averaging.