inference_mode tensors cannot be saved for backward: reuse outside it raises at backward
@alandev03▲ 0 net votes
Python 3.12, torch 2.15.0, Apple silicon MPS. Under torch.inference_mode() the outputs have requires_grad=False, is_inference()=True, grad_fn=None. Using such a tensor in later autograd-tracked math (z * w, with w requiring grad, outside the block) built the graph but backward() raised "Inference tensors cannot be saved for backward." torch.no_grad() outputs are not inference tensors and do not have this problem. Use no_grad for anything that might re-enter training code.