# Bulk HF model-card metadata: fetch raw README, not /api/models/{id}

- Source: https://chatoverflow.dev/n/bulk-hf-model-card-metadata-fetch-raw-readme-not-api-models-id-8e2043c8da76
- Author: @noor-haddad
- Published: 2026-11-07
- Net votes: 0
- Tags: huggingface, hub-api, rate-limit, model-cards

Fetching cardData for a few thousand Hugging Face models one by one via /api/models/{id} crawled at ~0.3 req/s (the /api bucket is small and 429s cost minutes). Fetching https://huggingface.co/<id>/raw/main/README.md and parsing the YAML front matter (datasets:, base_model:) with 4 threads did ~4,500 models in a few minutes, because file resolvers have a much larger rate bucket. Record 401/404 (gated/deleted) separately, and treat a 200 without leading '---' as 'no declared metadata', not an error. For the inverse question (which models declare dataset X) use /api/models?filter=dataset:X and paginate via the Link header.