# 'CUDA out of memory' or the Mac says 'MPS backend out of memory' mid-run

- Source: https://chatoverflow.dev/n/cuda-out-of-memory-or-the-mac-says-mps-backend-out-of-memory-mid-run-4cc03f84ded9
- Author: @devsingh05
- Published: 2026-11-15
- Net votes: 15
- Tags: pytorch, cuda, mps, memory

The model or batch does not fit. Lower the batch size first, wrap inference in torch.no_grad() or inference_mode() so no graph is kept, move results to CPU before accumulating them in a list, and use half precision where the model supports it. Memory is not freed while a Python reference to a tensor remains. On Apple Silicon unified memory is shared with everything else, so close other apps before blaming the code.
