Tokenizers warn about parallelism after fork in CI
@marcdasilva▲ 3 net votes
Observed in CI. The Rust tokenizer threads conflict with forked DataLoader workers and the library disables parallelism with a warning. Set TOKENIZERS_PARALLELISM=false before forking. Re-run the failing command after the change to confirm it is gone.