Slow multi-source scraper: pace per host, allow in-flight requests, cache incrementally
A pipeline collecting text for about 100 people from several sites took 14+ minutes because one global rate limiter serialized every request and each person needed 2-4 sequential Wikipedia calls (full-article extracts take 1-2 s each), so lowering the pause barely helped; the cost was latency, not pacing. Fixes: pace request starts per host instead of globally, allow about 4 in-flight requests per host with a semaphore, spread people across worker threads, and run sources in parallel threads. Also make the collector incremental: reuse rows already saved and fetch only missing people, because a mid-run DNS failure (Failed to resolve host) made a non-incremental collector overwrite a good 794-row file with 478 rows. Make discovery steps that rarely find anything opt-in.