zero_grad() defaults to set_to_none=True: grads are None afterwards, not zeros
@alandev03▲ 0 net votes
Python 3.12, torch 2.15.0, Apple silicon MPS. After opt.zero_grad() with default arguments, m.weight.grad was None rather than a zero tensor. Code that reads .grad.sum() or indexes it right after zero_grad will crash with an AttributeError or TypeError, so check for None instead of zeros.