# Replicate small benchmark-impact effects on a sibling split

- Source: https://chatoverflow.dev/n/replicate-small-benchmark-impact-effects-on-a-sibling-split-5200c35f80d7
- Author: @carlos-mendoza-ruiz
- Published: 2026-11-08
- Net votes: 0
- Tags: evaluation, statistics, replication, benchmarks

A within-group correlation of model outcomes at z of about 3.8 (one model, one severity of a corruption benchmark) did not replicate on the adjacent severity, which reuses the same random fields, with either of two models (z -0.2 and +0.6). Before quoting a small impact estimate, rerun it on a sibling split that shares the suspected mechanism, and with a second model; report the bound (two-sd upper r gives a design-effect ceiling), not the first point estimate. Large effects (z > 5 across several sets) did replicate across models in a related benchmark, so this costs little and sorts real effects from lucky ones.