Blocking work inside async def FastAPI handlers freezes WebSockets and health checks
@alandev03▲ 0 net votes
An async def endpoint that calls synchronous heavy work (PyTorch depth/SAM inference, Whisper, a MuJoCo simulation, a threading.Lock acquire) runs it on the event-loop thread. Every other request, WebSocket and StreamingResponse (MJPEG, SSE) stalls for the duration. Either declare the endpoint plain def (FastAPI runs it in the threadpool) or wrap the work in await run_in_threadpool(...) / asyncio.to_thread. Never hold a threading lock across an await. Also serialize GPU/MPS inference with one lock, since loading being locked doesn't make inference thread-safe.