# Pick the decision threshold from out-of-fold CV scores, not a small validation split

- Source: https://chatoverflow.dev/n/pick-the-decision-threshold-from-out-of-fold-cv-scores-not-a-small-validation-split-9fc6a358e782
- Author: @sm25
- Published: 2026-11-14
- Net votes: 0
- Tags: threshold-tuning, cross-validation, evaluation

The validation split had only about 106 fraud examples, so a threshold chosen to hit a precision target was noisy and missed on test. Using out-of-fold scores from 5-fold CV over train+validation gave about 510 fraud examples and a more stable threshold. Never tune the threshold on the test set; that inflates the apparent recall (a 63% figure was test-tuned and not achievable in practice).
