Torch worker_init_fn reseeding misses library-owned import-time generators
If a library keeps its own unseeded np.random.Generator created at import and uses it for seed=None, torch DataLoader workers forked from the parent all inherit that one state. torch's automatic per-worker seeding, and the classic worker_init_fn that reseeds np.random and random, only reach the global states, so they do not help: 8/32 distinct augmentations per epoch with 4 workers in our test, the same with the recipe. persistent_workers=True stops the epoch-to-epoch replay (8/24 over 3 epochs) but the workers still share one stream. What worked: in worker_init_fn, replace the library's generator with np.random.default_rng(torch.initial_seed() % 2**32), or pass seed= explicitly on every call (32/96). The replay across epochs also needs the parent never to draw from that generator between epochs; if it does, epochs differ but the workers still match each other. Check for this whenever a library migrated from np.random.* to a cached default_rng(): the migration removes the protection torch gave the old global-RNG code.
Reproduced on torch 2.15.0+cpu, NumPy 2.5.4, CPython 3.12.16 with a stand-in library module whose _rng = np.random.default_rng() is created at import and used when seed=None. 32 items, 4 workers, 3 epochs, fork: 8 distinct values out of 96, identical per item in all epochs (32/32). The classic worker_init_fn (np.random.seed + random.seed from torch.initial_seed()) changed nothing: 8/96 again. persistent_workers=True: 24/96, so no epoch replay, but the 4 workers still share one stream. Replacing the module's generator in worker_init_fn with default_rng(torch.initial_seed() % 2**32) gave 96/96.
Addition on scope: with multiprocessing_context='spawn' the same Dataset gave 96/96 without any fix, because each worker re-imports the library and builds a fresh unseeded generator. So the bug needs fork; I did not test 3.14's default forkserver (it preloads __main__, so it likely behaves like fork if __main__ imports the library).