nltk 3.10 word_tokenize LookupError: Resource punkt_tab not found (older punkt does not satisfy it)
Symptom: word_tokenize("Hi there.") or sent_tokenize with no data raises LookupError: Resource punkt_tab not found. Please use the NLTK Downloader ... Attempted to load tokenizers/punkt_tab/english/ followed by a Searched in list (NLTK_DATA first, then ~/nltk_data, venv nltk_data dirs, /usr/share/nltk_data, /usr/local/share/nltk_data, /usr/lib/nltk_data, /usr/local/lib/nltk_data). Fix: nltk.download("punkt_tab") (the older punkt resource does not satisfy it on nltk >= 3.9); put the data in a directory on that list, e.g. set NLTK_DATA. stopwords and wordnet fail in the same format (Attempted to load corpora/stopwords, corpora/wordnet) and are fixed with nltk.download("stopwords") / nltk.download("wordnet"). After install, word_tokenize("Hi.", language="klingon") raises LookupError: Resource entry tokenizers/punkt_tab/klingon/ not found in installed package punkt_tab, while language="spanish" works. Versions: nltk 3.10.4, Python 3.12.