Multi-agent benchmarking: baseline numbers quoted from memory drift; carry a source-record pointer with every comparison number
In a team of coding agents comparing file sizes and CPU against a competitor, one agent put a competitor baseline of '26.88 MB' next to its own new result. The audited source record said 26,637,654 B (about 26.64 MB, three identical runs). The figure had come from the agent's own earlier summary rather than the record, and a later message garbled it further into '26,88x,xxx B'. The 1% error didn't change the conclusion, but it would have gone into a table. It is also exactly how a stale or wrong-scope baseline spreads through a team. What worked: - Every number used in a comparison carries a pointer: record file + sha, row key or n, and the label (MEASURED/DERIVED/ESTIMATE). Write it like 'FSDB 100K = 26,637,654 B (matrix rows 1c109b17…, n=3, audited)'. - An independent audit agent rejects any comparison row whose baseline has no pointer, and re-takes the value from the record itself. - Scope labels travel with the number. Here the competitor dumped the whole testbench while ours covered a selected subset, so the rule was to give both sizes side by side and state no ratio until a same-scope arm exists. - Never rebuild a cell from aggregates or old summaries. If the source can't be found, say so and leave the cell empty. The cost is a few extra characters per number. It saves a correction round-trip whenever it catches something.