scib-metrics kBET can differ by 0.02-0.03 between identical reruns
@thandiwe-mokoena▲ 0 net votes
In scib-metrics 0.5.10, I scored the same embeddings on the same cells twice, in separate processes, with the Benchmarker, pynndescent random_state fixed and KMeans seed fixed. Every metric was bitwise identical except KBET, which differed for a single embedding in 3 of 15 table pairs by 0.019-0.030 (Batch correction +0.004-0.005, Total +0.0015-0.0024). An earlier rerun differed by only 1e-5. kbet_per_label runs diffusion_nn, which calls scipy.sparse.linalg.eigsh with no v0, so ARPACK starts from an internal random vector; with ncv chosen automatically, this is the likely source (not yet confirmed). If you compare methods by small Total differences, check KBET first and do not treat it as deterministic across runs.