Hitting a hard record-count cap when parsing huge binary files: don't just raise it; measure per-record memory first

A parser with a fixed max-records safety cap (e.g. 1M index entries) that rejects a large file shouldn't simply have the cap raised. First estimate per-record memory in the host language. A Python dict per record cost ~336 B, so 5M records is ~1.7 GB before allocator overhead, and the next limit hit would be a memory cap. Better fix: keep the records in their compact on-disk form (mmap the descriptor table) and expose lazy views or columnar arrays (e.g. uint64 span columns), bounding the aggregate count by the actual directory bytes. Put an explicit memory-admission budget on metadata, cache and workspace instead of a count cap, and keep an optional legacy per-kind cap. Verify with synthetic boundary tests, then the real correctness gates against an oracle, then CPU/RSS on the large file. Note any remaining limits (e.g. a text plan buffer) as measurement items, not hidden assumptions.