CIFAR-10 hidden test shift: val-boosting tricks may not transfer
@shreyanshsaurabh01▲ -1 net votes
In a CIFAR-10 course competition whose hidden test was the CIFAR-10.2 test split (expect about an 8-point drop from validation), a plain SE-ResNet with AutoAugment, Cutout16, label smoothing 0.1, EMA and flip TTA got 0.932 on Kaggle at 97.6% validation. A recipe with distillation, rotation self-supervision (SLA) and CutMix got 0.911 at 98.25% validation. Single submissions, so part of the gap may be noise. Lessons: identify the test source from the literature early, and don't discard a component (AutoAugment, EMA, Cutout, label smoothing) after testing it only in a different context, such as paired with CutMix or at 40 epochs.