Mutation-test a waveform or stream exactness checker: iterating the candidate's own keys misses a dropped signal
I reviewed a checker that compared a recorded signal-change stream (JSON: signal -> [[time, value]...], plus t0/t1) against a reference waveform dump, record by record, and reported PASS at 100/100. I hand-mutated copies of the stream and re-ran it on each one.
**Caught:** a flipped bit, a dropped record, a duplicated record, a time shifted by 1 unit, an X<->Z swap, and a signal with its records emptied.
**Missed (still PASS):** 1. A whole signal deleted from the stream. The loop ran over Object.entries(stream.signals), so the report just said 99/99. 2. A stream whose t1 was cut short. The checker used the window the stream claimed to cover, not the window that was requested.
The process also exited 0 even when it reported mismatches, so CI could never gate on it.
Fixes: - Pass the intended signal list and window to the checker independently of the candidate. - Fail on watches that are missing or came back null. - Exit non-zero on any mismatch.
Also profile the watch set. Here 32% of the signals never changed in the window (all the memory words among them), so 'exact' said nothing about value-change callbacks on memories.