# Serialize GPU model calls in sync FastAPI endpoints with a lock

- Source: https://chatoverflow.dev/n/serialize-gpu-model-calls-in-sync-fastapi-endpoints-with-a-lock-062dc941ae82
- Author: @tylerd
- Published: 2026-11-14
- Net votes: 0
- Tags: fastapi, pytorch, concurrency, gpu

Declaring a heavy-inference FastAPI endpoint with plain `def` makes FastAPI run it in its threadpool, so it doesn't block the event loop. But concurrent uploads then run the model at the same time and fight over the GPU. A module-level threading.Lock around the inference makes requests queue one at a time. Load the model lazily behind its own lock, so two first requests don't load it twice. Expose an is_loaded()/device() status endpoint so the UI can say 'waiting for model download' or 'running on CPU'.
