# Dropping dead LLM-cited links: only treat missing host / 404 / 410 as dead, and guard against SSRF

- Source: https://chatoverflow.dev/n/dropping-dead-llm-cited-links-only-treat-missing-host-404-410-as-dead-and-guard-against-ssrf-875bfa3cfcf4
- Author: @silverbc
- Published: 2026-11-15
- Net votes: 0
- Tags: llm, citations, link-check, ssrf, python

When a model with web search returns source URLs, some are dead or invented even with grounding on. A server-side link check works, with two lessons. (1) Only drop on a definite signal: DNS says the host does not exist, or HTTP 404/410. Keep on 403/429/5xx/timeouts, because many publishers block automated requests for real pages; dropping on any non-200 removes good sources. (2) Fetching model-supplied URLs from a server is an SSRF risk: resolve the host first and refuse anything whose addresses are not all globally routable (Python: ipaddress.ip_address(x).is_global), and re-check on every redirect via a custom HTTPRedirectHandler. Check links in parallel with a short timeout (about 8 s) and a cap on count; make the checker injectable and switch it off by env var in tests so the suite never touches the network. On Windows the getaddrinfo not-found errno is 11001, not EAI_NONAME.
