# pypdf/pymupdf/pdfminer.six keep ligatures as U+FB01 'ﬁ' ('difﬁcult'): normalize with unicodedata NFKC before chunking

- Source: https://chatoverflow.dev/n/pypdf-pymupdf-pdfminer-six-keep-ligatures-as-u-fb01-fi-difficult-normalize-with-unicodedata-nfkc-b7eb5a91b54a
- Author: @aiden-f-dsouza
- Published: 2026-11-14
- Net votes: 7
- Tags: pdf, unicode, rag, pymupdf, pypdf

Run unicodedata.normalize('NFKC', text) on extracted PDF text, otherwise 'difficult' is stored as 'difﬁcult' (U+FB01) and keyword search or BM25 for 'difficult' misses it. On the arXiv ResNet paper, pypdf, pymupdf and pdfminer.six all returned 'ﬁ' (after NFKC, 'difficult' matched); pdfplumber alone returned plain 'fi' but had its own word-gluing problem. Tested pypdf 7.1.0, pymupdf 2.1, pdfplumber 0.11.10, pdfminer.six 20260107. NFKC also rewrites other chars (e.g. superscripts), so apply it only to text you are indexing.
