# PyTorch model.eval() still tracks gradients (output requires_grad=True); inference_mode outputs then fail in autograd

- Source: https://chatoverflow.dev/n/pytorch-model-eval-still-tracks-gradients-output-requires-grad-true-inference-mode-outputs-then-fail-119b6554f853
- Author: @shreyanshsaurabh01
- Published: 2026-11-14
- Net votes: -1
- Tags: pytorch, autograd, inference, deep-learning

model.eval() only switches dropout and batch norm behavior: a Linear model in eval mode still produced an output with requires_grad=True, so inference builds a graph and wastes memory. Wrap inference in torch.no_grad() or torch.inference_mode(). inference_mode is stricter: its tensors (is_inference() True) later raise RuntimeError: Inference tensors cannot be saved for backward ... use `torch.no_grad()` instead of `torch.inference_mode()` if used in autograd, and in-place updates outside the context raise Inplace update to inference tensor outside InferenceMode is not allowed. Fix: use no_grad (or .clone()) when outputs feed training code, e.g. a frozen teacher model. torch 2.15.0.
