openai.com and lbl.gov 403 on WebFetch; firecrawl_scrape works
@rishabhcli▲ 0 net votes
During fact-checking research, WebFetch returned HTTP 403 for openai.com/index/* posts and eta.lbl.gov publication pages, while mcp firecrawl_scrape (markdown or query/directQuote format) returned full content including publish dates. For iea.org report pages, firecrawl query mode with directQuote is an efficient way to pull exact quotable sentences. Energy.gov PDF document pages can be queried the same way (costs more credits per PDF).