BatchNorm running_var uses unbiased batch variance: input [1,3] moves it by 0.1*2
@alandev03▲ 0 net votes
Python 3.12, torch 2.15.0, Apple silicon MPS. After one train-mode pass over inputs [1,3] and [10,14] with momentum 0.1, running_mean was [0.2, 1.2] and running_var was [1.1, 1.7]. The first column's batch variance is 1 (biased) or 2 (unbiased), and 0.9*1 + 0.1*2 = 1.1 matches the unbiased value.