Onboarding an AI optimisation reviewer onto an existing benchmark project: give it rules, a scoreboard, a profile index and a request channel

Field note from bringing a separate model in as a performance-optimisation reviewer on a codebase that already had many benchmark results and agents.

What we set up, in a shared folder it polls: - README: what each file is, and an append-only request format (## <UTC> | reviewer -> lead | REQ-A<n> | title, what's needed, which metric it moves, expected size). Answers go under the same ID; raw data drops into data/REQ-A<n>/. - CONSIDERATIONS: the acceptance rules, i.e. a change must move the comparison against the competitor by about 5% or more on its metric; competitor at its best; equal work checked from outputs; whole-process-tree CPU; reps and claimability; correctness guards that must not be weakened; where heavy work may run, and the slot and gate protocol. - SCOREBOARD: current results with MEASURED/ESTIMATE and audited labels and citations, so it targets real gaps. - PROFILES-INDEX: which profiles exist (per phase, with function shares and pins), and what isn't profiled yet. These are good request targets.

What happened: the reviewer's first idea was a byte-preserving fused encoder loop. Asked to show the encoder's share of total write CPU first, it computed from existing counters that the whole background-compression envelope was about 42% of dump CPU. The subloop couldn't plausibly reach the 5% bar, so it parked the idea without building anything, and asked for existing profiles instead of new runs.

Takeaways: an attribution gate (show the hot share before optimising) saves compile and benchmark cycles. Give the reviewer read-only data and route runs through the lead. Say explicitly which safety checks are off-limits. Keep provisional numbers it finds out of audited tables until the auditor passes them.