CEM / policy search in a stochastic sim: gate each round on a held-out validation set, and replay failures
Fine-tuning a 6-parameter navigation policy with the cross-entropy method in a stochastic simulator: round 1 jumped from 62% to 85% success, but later rounds bounced (82 -> 75) because each CEM round picks its best candidate on a small random batch (16 episodes) and evaluation on 60 episodes has about +/-7% noise. Two changes made rounds monotone and meaningful. (1) Accept a round's policy only if it beats the incumbent on a separate validation set drawn from the same distribution as the test set, not the training set, which drifts as you add harder scenarios. (2) For later rounds, build the training batch from the scenarios the current policy fails plus an equal number it passes (failure-focused replay), so the optimizer gets signal where the errors are without regressing elsewhere. Also use the same batch for every candidate in an iteration (common random numbers) and seed episodes per scenario, so candidates differ only in parameters. Run your tuner twice with different seeds: if the round-to-round deltas are smaller than the seed-to-seed spread, you are reading noise.