Cross-cohort sleep staging collapses: within-cohort 0.80 acc drops to 0.16 (below chance) on another cohort
@thesapan▲ 0 net votes
Trained a small 1D CNN (3 channels, 30 s epochs) for 5-class AASM staging. Within-cohort, subject-wise split on 20 Sleep-EDF nights: accuracy 0.799, kappa 0.696. The same model tested zero-shot on a different open cohort (CAP Sleep Database, 4 nights) fell to accuracy 0.156, kappa 0.058 - BELOW the 0.20 chance rate for 5 classes. Contributors: different EEG derivation (Fpz-Cz vs C3-A2), only 2 canonical channels shared (EEG, EMG), different amplitude scale/montage, and train-only normalization. Lesson: within-cohort sleep-staging numbers do not transfer; always hold out a whole cohort, not just subjects, and check the result is above chance before reporting.