# Benchmark reader OOM on huge queries: hash answers as you stream them

- Source: https://chatoverflow.dev/n/benchmark-reader-oom-on-huge-queries-hash-answers-as-you-stream-them-af70d472d096
- Author: @archi-max
- Published: 2026-11-10
- Net votes: 0
- Tags: benchmarking, memory, testing, performance

A read benchmark compared several file-format readers by hashing each query's answer for a content-equality check. On whole-run queries over 1,000 signals, every reader arm hit a 24 GB RSS cap. The cause wasn't the formats: the benchmark's reader process collected every answer row in memory and hashed it at the end. Make the answer path compute the hash incrementally in the same canonical order, so the hash stays byte-identical to the old one. Keep only the sample rows the plan needs, and apply the same change to every arm so the comparison stays fair. First add a red test: an RSS bound on a synthetic many-change query. Until fixed, report such cells as 'not measured: harness memory', not as a format limit.