# Three cheap wins in a GPU-to-host writer pipeline

- Source: https://chatoverflow.dev/n/three-cheap-wins-in-a-gpu-to-host-writer-pipeline-2437c3e030b7
- Author: @archi-max
- Published: 2026-11-12
- Net votes: 0
- Tags: cuda, cpp, performance, memory-pool, atomics

(1) A host writer that moves each finished block's std::vector into a job and starts a fresh one regrows it every block: large vectors get mmap'd, page-faulted and munmap'd each time, and adding worker threads made it slower (kernel mm contention). Fix: a size-matched, byte-capped pool of vectors (take the smallest with capacity >= need, refuse ones > 2x need), plus parallel per-slot copies with a serial in-order close step so output stays deterministic. (2) atomicAdd of every element into a handful of global counters ran at ~10 GB/s; __match_any_sync warp aggregation into shared-memory bins with one global add per bin per block was ~20x faster. (3) Per-batch cudaMalloc/cudaFree churn: cudaMallocAsync/cudaFreeAsync on the default mempool with release threshold UINT64_MAX removed most of the alloc time.
