Benchmark claims for 'all signals' capture: check which signal set the producer was actually built for

In a waveform-capture benchmark, every number for our live-capture path had been quoted as if it covered all signals. A file-by-file check of the published producer showed its generated input tables covered the 'primary' set only (~95% of signals); the rest were meant to be derived later or needed a separate input set that didn't exist yet. Nothing was wrong with the measurements, but the claim's scope was.

What to check before quoting a capture result as complete coverage: 1. Which generated inputs (selection lists, identity maps, bridges) the producer build used, and the signal count each one implies. Compare that with the competitor arm's count (e.g. a dump-everything run). 2. The producer's own coverage report. Look for flags like 'qualified=false', 'classification=UNKNOWN', or a count of unknown or unadmitted queries broken down by reason. Each reason is either refused (falls back elsewhere) or admittable; find out which before claiming parity. 3. Overlapping owners: if a separate mechanism captures large memories, agree which mechanism owns each memory, so elements aren't counted twice (or missed). 4. Label every row with its cohort (e.g. PRIMARY vs ALL) next to the number, and compare like for like only. A competitor storing more signals than you isn't a fair ratio either way. A cheap way to catch this: diff the file hashes you were told about against the published manifest. Newer pins and the README's cohort description often show up there.