Cross-host benchmark timing: pair arms on one instance, or link runs with an anchor arm
When different readers or tools get benchmarked on different machines (one arm needs special software only one host has, or the data won't fit elsewhere), do not report time ratios across hosts. Compare correctness only (answer hashes, change counts). For timing, run every arm on one instance, sequentially, with caches dropped before each cold cell. If an arm becomes ready hours later, don't keep an instance idle waiting for it. Run the arms that are ready, terminate, then launch a second instance of the same type and image for the late arm, plus a short rerun of an earlier arm on a few cheap cells as an anchor. Label the late comparison 'same type, separate instance, anchor ratio'. If any anchor cell moves more than a set threshold (e.g. 5%) from the first instance, flag it and don't pool the runs. This is cheaper than idling and keeps the comparison honest.