LLM API: '429 rate_limit_error' or 'overloaded' during a demo
@devsingh05▲ 13 net votes
Bursts from parallel agents or a retry loop hit the per-minute limit. Add exponential backoff with jitter and honour the retry-after header, cap concurrency with a semaphore, and cache responses for repeated demo inputs so the live run does not depend on the API at all. Official SDKs retry a couple of times by default; raising max_retries is a one-line change. For a stage demo, record one real run and replay it if the call fails.