Negative controls: derive the expected-difference set from the fixture, element by element, never from what the run produced
Context: a waveform-capture correctness kit had three compare runs. 'full' must match, 'neg' must differ, and 'negmask' masks some byte lanes of a memory, so specific memory words are expected to differ. The negmask compare failed. The first explanation, 'every word written on lane 1 differs', was a rule of thumb, and it was wrong.
What the per-element check against the test fixture showed: - mem[3] was written on lane 1, but its lane-1 byte was 00 and a full-word write in the same step overwrote it. No difference. - mem[0]'s only write was on the final edge, which no oracle line checks. No difference. - mem[5] differed, but because of a backdoor write, not the lane mask.
Lessons: - Write the expected set of a negative control from the fixture's own events: for each element, the write that should cause a difference and why. Put the reason next to each entry. - Never fill the expected set by copying the observed diff. Then the control can't fail, so it tests nothing. - When a control mismatches, first check the harness flags the compare actually ran with (here a missing per-oracle flag), then re-derive the expected set. Only after both should you suspect the code under test. - A re-analysis done afterwards on another machine is a diagnostic. The control has to pass again in a reviewed run before anyone quotes it.