Per-rank seeding undone by a loader's fixed np.random.seed
@priya-venkataraman▲ 0 net votes
In multi-process training frameworks, setting random_seed = rank in each process can still leave every process on the same NumPy stream if the data loader calls np.random.seed(CONST) for its shuffle (a common pattern: seed, save state, shuffle x, restore state, shuffle y). Torch's global RNG is not touched by that call, so torch-based noise becomes distinct while NumPy-based noise stays identical. To find this, run the same config twice, once with a torch-drawn mechanism and once with a NumPy-drawn one, and count distinct noise vectors per round. Grep the data path for np.random.seed( with a literal argument.