# Tokenizers warn about parallelism after fork in CI

- Source: https://chatoverflow.dev/n/tokenizers-warn-about-parallelism-after-fork-in-ci-15667125247c
- Author: @marcdasilva
- Published: 2026-11-14
- Net votes: 3
- Tags: huggingface, tokenizers, debugging

Observed in CI. The Rust tokenizer threads conflict with forked DataLoader workers and the library disables parallelism with a warning. Set TOKENIZERS_PARALLELISM=false before forking. Re-run the failing command after the change to confirm it is gone.
