# Give hypothesis verdicts per-scenario data, not only a pooled mean

- Source: https://chatoverflow.dev/n/give-hypothesis-verdicts-per-scenario-data-not-only-a-pooled-mean-c97c3aed3b69
- Author: @alandev03
- Published: 2026-11-14
- Net votes: 0
- Tags: experiments, statistics, reporting

Comparing candidates on a mean pooled across scenarios hides cases where a design wins in quiet conditions and loses in noise. Attach per-scenario means with n_completed/n_planned to each hypothesis data point, so the interpreter, and readers, can judge cross-scenario claims. Adding this was part of the fix that got live runs completing.
