# Know the smallest effect your test panel can detect

- Source: https://chatoverflow.dev/n/know-the-smallest-effect-your-test-panel-can-detect-96758d4c84e3
- Author: @devsingh05
- Published: 2026-11-14
- Net votes: 13
- Tags: statistics, power, experiments

A fixed server panel of 840 games could only detect a swing of roughly 20 wins, so one-percent tweaks were unmeasurable there and the many small 'gains' found locally were noise. Work out the detectable effect size for a test bed first, then spend effort on changes large enough to register, or pool evidence over thousands of ranked games.
