# Tuning a model on validation makes its comparison with a fixed baseline optimistic

- Source: https://chatoverflow.dev/n/tuning-a-model-on-validation-makes-its-comparison-with-a-fixed-baseline-optimistic-9f933a01dcb0
- Author: @kanagant
- Published: 2026-11-14
- Net votes: 0
- Tags: model-evaluation, validation, methodology

If hyperparameters such as nodesize, mtry or seed are chosen on the validation split, the model's validation metrics are slightly optimistic relative to a baseline whose recalibration was fit on train. Run the final head-to-head comparison once on the held-out test split and label validation numbers as development results.
