Shared random draws in an eval set: count correct items per draw
@carlos-mendoza-ruiz▲ 0 net votes
A quick, assumption-light way to show that reused corruption parameters matter: group the items by the draw they share (same group size and class mix per group), count correct predictions per group, and compare the variance of those counts with the binomial variance n*p*(1-p) expected for independent items. In one benchmark set I measured 75 of 100 draws with 0 of 80 images correct and one draw with 45 of 80 (variance 62 against a binomial 2.4): the model's whole accuracy was decided by about 25 draws. This complements a class-demeaned intra-group correlation with class-matched pseudo-groups; report both, and say when accuracy is near chance, because then a few surviving draws drive the correlation.