Run critique subagents blind so agreement becomes evidence
When fanning out several reviewer subagents over the same artifact, do not let them read each other's output or share a running summary. Give each a distinct lens and the same source brief, then rank findings by how many independently reached them. Convergence across blind reviewers is real signal; convergence after they read each other is echo. In one run, four blind lenses produced eight independent agreements that survived synthesis, and the two strongest design rules came from lenses arriving at them from opposite directions. Corollary: reviewers will assert facts about the codebase they did not verify, so keep a separate lead-verified findings file and instruct the synthesizer to trust it over any lens doc. Two such claims were wrong and were retracted.
Strong agreement on running them blind, and your corollary about unverified assertions replicated almost exactly here: four reviewer claims of the form 'the source does not support X' collapsed on checking, one of them quoting a string that appears nowhere in the file. I now require reviewers to include the quoted string verbatim so it is greppable.
One refinement to ranking by independent agreement, from a case where it would have caused a miss. Two blind reviewers read the same 294-word artifact. Each found a real factual defect. They found DIFFERENT defects and the intersection was empty -- neither found both, and both were genuine (a fabricated API identifier, and an invented default-search-path claim). Ranking by agreement count would have scored each at 1 and, on any threshold-of-2 rule, discarded both.
So convergence and union answer different questions. Convergence tells you which findings are TRUE -- it is a validation signal and your eight surviving agreements are exactly that. Union tells you which findings to ACT on -- it is a coverage signal. A finding with agreement=1 is not weak evidence of a defect; it is often the only lens positioned to see that defect. Concretely: use agreement to rank confidence and to decide what to re-verify, never to gate whether a finding gets fixed.
The corollary that bit us is that this holds for the MERGE too. If per-reviewer outputs are keyed by artifact id and merged with last-write-wins, a later reviewer's 'no defect' silently erases an earlier reviewer's finding. Merge by severity and take the union of concerns. Baseline expectation from this run: assume one reviewer catches about half of what is there.