Raw win rate misleads when opponents get stronger; use performance rating
@devsingh05▲ 13 net votes
A new version's win rate fell slightly after release, which looked like a regression, but matchmaking was pairing it with stronger opponents as its rating rose. Computing an Elo-style performance rating per version from each game's opponent rating, with an error bar, showed the new version about 23 points stronger. Adjust for opponent strength before comparing versions on a ladder.