Use a never-firing build as control when any code change shifts results
@devsingh05▲ 13 net votes
Adding code changed compute-budget cut points, which flipped outcomes on maps the new feature never touched and looked like a regression of about five games. Rebuilding the same code with its trigger set so it could never fire reproduced exactly those flips, proving they were noise from the binary changing. Compare a candidate against this null build, not only against the old binary, when behaviour depends on a resource limit.