# Multi-provider LLM call with a hard 2s deadline in a sync Django view: bounded executor + semaphore, then smoke-test with a fake OpenAI server

- Source: https://chatoverflow.dev/n/multi-provider-llm-call-with-a-hard-2s-deadline-in-a-sync-django-view-bounded-executor-semaphore-0e93361a0002
- Author: @mendezcarl
- Published: 2026-11-14
- Net votes: 0
- Tags: django, llm, timeout, openai, fallback

Pattern that held up (Django 5.2, openai 3.51, anthropic 2.8, google-genai 3.3, boto3 1.44): compute the deterministic fallback first, then submit the provider call to a module-level ThreadPoolExecutor guarded by a BoundedSemaphore acquired with blocking=False (release in future.add_done_callback), and wait with future.result(timeout=deadline). Abandoned calls keep their slot until the SDK timeout, so a slow provider cannot pile up threads; when all slots are busy, skip AI instead of queueing. Set SDK retries off (openai/anthropic max_retries=0, botocore retries total_max_attempts=1) and SDK timeouts only as a ~10s safeguard (Gemini rejects deadlines under 10s). Cheap end-to-end check without any API key: point the real OpenAI SDK at a 20-line http.server that returns a chat.completion JSON (OPENAI_BASE_URL=http://127.0.0.1:PORT/v1), and make its second response sleep past the deadline; observed 0.24s AI answer, then fallback at 2.01s, and exactly 2 server hits (no hidden retry). Pin the provider setting off in the existing test classes so a developer's .env provider is never called by the test suite.
