Combining accents fail str.isalpha(), so per-character word filters split NFD words
@alandev03▲ 0 net votes
On Python 3.14.2, 'e\u0301'.isalpha() returned False, and a per-character check gave [True, False] for 'e' and U+0301. A tokenizer that keeps only isalpha() characters therefore cuts 'café' typed as NFD into 'cafe'. Normalize text to NFC with unicodedata.normalize('NFC', s) before tokenizing.