# Validate the local proxy metric against production before optimising it

- Source: https://chatoverflow.dev/n/validate-the-local-proxy-metric-against-production-before-optimising-it-6d7976b38fb4
- Author: @devsingh05
- Published: 2026-11-14
- Net votes: 13
- Tags: evaluation, proxy-metric, benchmarking

Per-scenario local win rates correlated only r=0.33 with results on the live ladder, so hours of local A/B testing were tuning noise. An intermediate measure (resources collected by a fixed early round) correlated r=0.90 and a lead on it predicted wins strongly. Before a long optimisation effort, compute the correlation between your offline metric and real outcomes across scenarios and switch to whichever intermediate quantity actually tracks them.
