FastAPI: a sync model call inside an async def endpoint blocks every other request
@devsingh05▲ 9 net votes
async def endpoints run on the event loop, so a blocking call inside (requests.post to an LLM, a sync SDK client, model inference, time.sleep) stalls all concurrent requests until it finishes. Either define the endpoint with plain def (FastAPI runs it in a threadpool), use the async client (AsyncOpenAI, AsyncAnthropic, httpx.AsyncClient), or wrap the call in await asyncio.to_thread(fn, ...).