Converter fails above ~65,500 signals on a default Linux box: one mmap per signal hits vm.max_map_count
Symptom: a waveform converter (VCD → custom format) failed in its packing step with a bare "cannot read" on a record file. The run had read all ~84K signals fine and the disk had plenty of free space. Three different inputs (80-85K signals each) all died at exactly the same record index, 65,521.
Cause: the packing step kept one memory mapping (mmap) open per signal until the end. Linux limits mappings per process with vm.max_map_count, default 65,530. The tool swallowed the real errno (ENOMEM from mmap), so it looked like an I/O error.
Why it hid for weeks: the team's main dev host had vm.max_map_count raised to 1,048,576, so the same signal counts passed there. Fresh cloud boxes and customer machines keep the default.
Checklist: - If a large-input job fails at the SAME index every time, suspect a per-object resource limit (open files: ulimit -n; mappings: /proc/sys/vm/max_map_count; threads), not data or disk. - Check with: cat /proc/sys/vm/max_map_count; ulimit -n; wc -l /proc/<pid>/maps during the run. - Workaround: sysctl -w vm.max_map_count=1048576 (root; per box). Fix: don't hold one mapping or fd per record. Batch, reuse, or close as you go. - Always print strerror(errno) on map/open failures. - Run scale tests (70K, 200K objects) on a box with DEFAULT limits, not your tuned dev host. - A related variant: one open file per signal hits ulimit -n (often 1,024 or 65,536) at a lower count.