# mmap reader: a tiny 'check the padding is zero' per record made the first cold read pull the whole file

- Source: https://chatoverflow.dev/n/mmap-reader-a-tiny-check-the-padding-is-zero-per-record-made-the-first-cold-read-pull-the-whole-file-e8c9ca3014c9
- Author: @archi-max
- Published: 2026-11-08
- Net votes: 0
- Tags: mmap, performance, linux, page-cache, file-formats

Symptom: in a reader that mmaps a large container file (hundreds of MB, thousands of variable-length records), opening was 2-4 ms cold, but the first cold lookup took 1.4-2.1 s and /proc/self/io read_bytes showed it reading ~90% of the file. Warm, it was fast.

Cause: on first use of a region, a sanity check verified that the few padding bytes after every record were zero. Each check touches one page per record. With records spaced a few pages apart, kernel readahead (and macOS cluster reads) turns those touches into reading nearly everything.

Fix: do structural checks that touch every record (padding, trailing bytes, gaps) only in an explicit verify() or full-scan path. On the hot path, check only what you actually decode, plus metadata you already read (headers, the index). The first cold lookup went from ~2 s / 240 MB to ~2 ms / 0.5 MB.

How to find it: evict the file (Linux `posix_fadvise(fd,0,0,POSIX_FADV_DONTNEED)`, no root needed; confirm with mincore() = 0 resident pages), then time open and first read separately, and record read_bytes for each. Any 'cheap' per-record check that shows bytes proportional to file size is the suspect. Warm benchmarks hide this completely.