nvCOMP batched zstd/Deflate in practice: no level knob, ~1.2x larger than CPU zstd-3, and keep your own checksum
Field notes from using NVIDIA nvCOMP (5.0 and 6.0.0.12) batched APIs on an L4. The data was delta-coded time/value rows, i.e. compressible structured records.
- zstd compress has no compression-level option in either version. You get one fixed effort. On our data, output was ~1.19x the size of CPU libzstd level 3 (single diagnostic run). If size matters, plan another route: build blocks on the GPU and compress with CPU zstd at a higher level (in our data z9 was ~0.83x of z3), or accept the size cost for a v0. - Each chunk becomes an independent standard frame, so CPU libzstd/zlib can decode GPU output. Use that as the correctness gate: strict-decode every GPU frame on the CPU and compare decoded content. Do not require byte equality with CPU-compressed frames. - Parallelism comes from many chunks in flight, not from within one chunk. There is no cross-chunk history, so chunk size trades ratio against parallelism; measure both for your block size. The zstd compress chunk limit is about 16 MB, and scratch memory is sized via the GetTempSize calls. - nvCOMP has no zlib wrapper for Deflate. Strip the 2-byte zlib header and the adler32 trailer yourself, feed raw deflate, and verify adler32 in your own kernel. - Batched Deflate decompress returned a success status on a deliberately corrupted stream (a 2-byte flip). Only our own adler32 check caught it. Never trust the per-chunk status alone; keep an independent checksum. - Inputs needed 4-byte alignment in our setup.