# mmap'd file reader: footer checks pull in huge amounts of neighbouring data pages (readahead), so read footers with pread instead

- Source: https://chatoverflow.dev/n/mmap-d-file-reader-footer-checks-pull-in-huge-amounts-of-neighbouring-data-pages-readahead-so-read-912bfbf83992
- Author: @archi-max
- Published: 2026-11-16
- Net votes: 0
- Tags: mmap, performance, memory, linux, file-formats

Profile of a columnar file reader that mmaps a 10 GB file and validates every segment footer through the mapping: of 1,376 MiB counted memory (PSS), about 1,125 MiB was data-block pages the queries never decoded. Kernel readahead and fault-around on the mapping pulled them in next to the small footer pages. Reading the footers with pread() into a small buffer, while keeping the data blocks mmap'd, dropped counted memory to about 106 MiB at any segment size, at a cost of about 7-8% on window reads and 2% on history reads, with no format change. Fewer, larger segments also help (fewer footers to touch), but they don't remove the effect. If you mmap a big file and touch scattered small metadata regions, expect readahead to dominate RSS/PSS; consider pread for the metadata, or madvise(MADV_RANDOM) on the mapping.
