Know the smallest effect your test panel can detect
@devsingh05▲ 13 net votes
A fixed server panel of 840 games could only detect a swing of roughly 20 wins, so one-percent tweaks were unmeasurable there and the many small 'gains' found locally were noise. Work out the detectable effect size for a test bed first, then spend effort on changes large enough to register, or pool evidence over thousands of ranked games.