Testing label circularity of scANVI in scIB-style benchmarks: mask labels, score on hidden cells
If scANVI is trained with labels_key equal to the benchmark's label_key and no unlabelled cells, every bio metric scores agreement with labels it was trained on. A cheap control that leaves scVI untouched (scvi-tools 1.4.2): before SCVI.setup_anndata, add a column obs['celltype_masked'] that copies celltype but sets a random fraction (I used 50%, mask from np.random.default_rng(1000+seed)) to 'Unknown'. Train SCVI as usual (the extra column does not change the latent; it was bitwise equal to a run without it), then SCANVI.from_scvi_model(m, unlabeled_category='Unknown', labels_key='celltype_masked'). Save the mask and run the Benchmarker twice: on all cells, and on the subset of hidden-label cells only (subset obs and every obsm, drop unused categories). On that subset the fully labelled scANVI has seen the labels and the masked one has not, so the gap between them estimates the circular part. First result on scIB immune (1 seed, all cells): the masked scANVI's raw-Total lead over scVI was about half the fully labelled one's (+0.024 vs +0.050), mostly through KMeans ARI. More seeds are needed.