Framed-copy-then-write doubles a writer's flush peak; stream it with writev and an incremental hash

Pattern seen in a file writer whose flush built (1) a body buffer by copying queued compressed chunks, then (2) a framed copy (header + body + trailer) for one write() call and a one-shot checksum. A flush of R queued bytes therefore needed about 2R extra on top of the queue: process RSS for the tool was about 500 MB against about 260 MB after the fix.

Fix, byte-identical by construction: - Describe the output as a list of pieces (header, each chunk in place, small zero pads, the small table you still build, the trailer) and write them with writev in batches (e.g. 64 iovecs per call, under IOV_MAX). Handle short writes by advancing into the current iovec, and skip zero-length entries. - Hash with an incremental XXH64 (or whichever checksum the format uses), updating per piece before that batch is written. Only the trailer, which carries the hash, is written last. - Test the incremental hash against the one-shot on random splits, with empty and 1-byte updates, and (opt-in, since it needs RAM) on a stream past 4 GiB. - Check identity on real outputs: byte-compare old vs new files across many cases and option sets, not just round trips.

Gotchas: - With UBSan, a test that hashes std::vector<uint8_t>().data() (nullptr) with len 0 trips 'applying offset to null pointer' inside the one-shot hash loop. Hash from a non-null buffer in tests. - A red test that measures 'extra memory during the flush' must take R as what the flush gives back. Thread-local scratch that stays allocated across the flush otherwise dominates the before-flush number and makes the check pass vacuously.