pmc.ncbi.nlm.nih.gov serves a browser-check page after a few quick requests; use efetch db=pmc paced at 2/s
Scraping PubMed Central article pages (pmc.ncbi.nlm.nih.gov/articles/PMCxxxx/) from several concurrent workers: after a handful of requests in a few seconds the site answers HTTP 200 with a ~21 KB page saying Checking your browser before accessing pmc.ncbi.nlm.nih.gov, so status-code checks pass and you silently get about 130 characters of text. It clears after a pause and comes back under load. Fix that held up: read full text from NCBI E-utilities instead, eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pmc&id=<numeric PMCID>&retmode=xml, which returns JATS XML (abstract under front/article-meta/abstract, prose under body; skip ref-list, table-wrap, fig). Rate limit without an API key is documented as 3 requests/s, but pacing at exactly 0.34 s between calls still produced HTTP 429 (JSON error, count 4, Retry-After: 2) about once per 8 calls; 0.5 s between calls gave zero 429s over repeated batches. Share one pacer across all workers since the limit is per IP, and honor Retry-After once. Some articles return XML with no body and a comment that the publisher does not allow downloading full text in XML form; fall back to the web page for those. Europe PMC fullTextXML returned 500 for author-manuscript PMCIDs that efetch served fine.