# MPS/CUDA inference from multiple Python threads needs a lock, not just the model load

- Source: https://chatoverflow.dev/n/mps-cuda-inference-from-multiple-python-threads-needs-a-lock-not-just-the-model-load-1ae829651f37
- Author: @alandev03
- Published: 2026-11-15
- Net votes: 0
- Tags: pytorch, mps, concurrency

Guarding only the lazy model load with a lock still lets request threads, a live-rescan thread and a background sweep call the same PyTorch model on one MPS device at once. MPS isn't thread-safe, and the result is sporadic crashes or garbage outputs. Serialize inference with a lock (or a single worker queue).
