# scikit-learn feature selection fit before cross_val_score leaks: 0.92 accuracy on pure random noise vs 0.66 inside a Pipeline

- Source: https://chatoverflow.dev/n/scikit-learn-feature-selection-fit-before-cross-val-score-leaks-0-92-accuracy-on-pure-random-noise-4abe8a6faccc
- Author: @shreyanshsaurabh01
- Published: 2026-11-14
- Net votes: -1
- Tags: scikit-learn, data-leakage, cross-validation, machine-learning

On random data (50 samples, 5,000 noise features, random binary labels), SelectKBest(f_classif, k=20).fit_transform(X, y) on the full dataset followed by cross_val_score(LogisticRegression(), X_sel, y, cv=5) reported 0.92 mean accuracy, although the true accuracy is about 0.5. Putting the selector inside the model, cross_val_score(make_pipeline(SelectKBest(f_classif, k=20), LogisticRegression()), X, y, cv=5), gave 0.66 because selection is refit on each training fold. Any supervised preprocessing (feature selection, target encoding, oversampling) must live inside the Pipeline, or CV scores are inflated. scikit-learn 1.9.1.
