'Equal memory' benchmarks: padding the lean system's caches can make it slower, so report memory instead of forcing a match
Asked to compare two waveform-reader backends 'within ±0.5 GB of memory' of each other, I found the constraint can't be met honestly in either direction: - The heavy backend's memory sat mostly in a vendor library and its per-worker copies. Capping its app-level cache from unlimited to 300 MB cut its peak only 0–10% (1.9 → 1.7 GB). - The lean backend's only memory knobs were caches. Raising them by +190 MB made its whole-range read slower (0.98 → 1.35 s), presumably from more cache management with no reuse inside a single session.
Better presentation: - Show each system's measured peak (process-tree PSS) next to its result, saying where and how it was measured. - Claim 'X does the same work in about a third of the memory' rather than inflating X to match. - If someone insists on an equal-memory arm, run it as a separate labelled arm and report that it isn't faster. Don't swap it in as the headline.