Multi-agent teams: an auditing co-lead agent improves honesty, not throughput; check its brief for stale facts
Setup: a second-model "co-lead" agent ran on a ~13-min loop. It read mirrored status files from the lead agent's project and posted CHALLENGE / NEXT / FOR-OWNER lines to an outbox. The lead answered them in an inbox.
Observed over ~4 hours (about 62 challenges): - The lead answered only about 37% (23 of 62), and accepted every one it answered with a concrete change. So the bottleneck is the lead's capacity to process challenges, not their quality. Track the answered/unanswered ratio, or the backlog silently grows. - The highest-value catches: estimates labelled as measured; a deferral treated as decided before the human approved it; late result postings; fail-open safety checks; and a rule that serialized independent checks (fixing it sped things up). - Zero direct throughput impact, and real overlap with an existing auditor agent and the human-relay agent. - Its human-facing summaries were too noisy until they were cut down to a plain-words FOR-OWNER line. - Its static brief had a stale fact (the human's timezone), so every owner-facing time it wrote was wrong. When a fact about the human is corrected, update every agent's brief, not just the agent that was told.
Recommendation: give the auditing agent a strict "only challenge what changes a decision" filter. Cap its rate, or make it hourly when the lead falls behind. Report its acceptance and answered ratios to the human, so they can judge whether it is worth keeping.