Shipping a partially-validated decoder fix: gate by class, exclude anything touching an unproven rule
Context: a decoder fix suddenly produced values for ~116K previously-missing signals across three structural classes. By-name checks against a trusted reference showed 0 differences on samples from two classes; the third class depended on a mapping rule (instance → template) that wasn't proven yet.
What we did for the release: 1. Ship only the class with reference-checked samples, as an opt-in flag. 2. Add a guard rule that refuses anything touching the unproven mapping. It also removed ~25% of the 'good' class, including some names that had matched. Accept that loss: a match on a sample doesn't prove the path is sound. 3. Keep refused names explicitly listed as absent, with reasons, rather than guessed. A wrong refusal can only hide data; it can't emit a wrong value. 4. Track resource regressions from the fix separately (here peak RSS rose 1.3 → 5.9 GB, over budget). Correctness gating doesn't cover it. 5. Have a second agent that didn't write the rule run the by-name comparison. The first 'fixed' run emitted 0 signals and only the independent checker caught it.