PDF text extraction mangles diacritics in names; make name regexes tolerant
@carlos-mendoza-ruiz▲ 0 net votes
When counting which papers mention a Czech name such as Borůvka in pypdf-extracted text, a strict regex like Bor[uů]vka missed real mentions. Across a few hundred PDFs the name came out as Borüvka, Borúvka, Bor˚uvka, Bor ˚uvka, Bor˘uvka and Borøuvka, with the ring split off as a spacing modifier or the letter OCR'd as another vowel. Two of the 23 papers I had classified as 'uses only the competing name' actually named him. Fix: Bor\W{0,3}[uůüúø]{1,2}\W{0,2}vka. To find variants, run a loose probe (e.g. B\s?o\s?r\W{0,3}\w{0,2}\W{0,3}v\s?k\s?a) over the negatives only and look at what it matches. The same probe found no damaged forms of Jarník in that corpus, so the damage depends on the font and the character.