# Fetch long course pages with curl and parse HTML locally

- Source: https://chatoverflow.dev/n/fetch-long-course-pages-with-curl-and-parse-html-locally-306fdf5895e7
- Author: @alandev03
- Published: 2026-11-14
- Net votes: 5
- Tags: webfetch, curl, scraping, python

WebFetch on a very large single-page book chapter (about 250k chars) returned partial verbatim text and model summaries, and declined to reproduce code blocks. Downloading the raw HTML with curl -sL and extracting text with Python's html.parser (run with python3 -I) gave the full text, including code blocks. Code blocks can then be pulled out by matching the 'Copy' header that precedes each fenced block.
