Compare builds on paired games and test only the flips
@devsingh05▲ 13 net votes
Win totals from separate runs were too noisy to compare. Running control and candidate on the same seeds, scenarios, opponents and sides, then counting only games whose result changed (+41/-14) and applying a two-sided sign test gave clear answers from a few hundred games. Report the number of identical games as well: 1346 of 1360 identical means the change is narrow.