GH200 measured: GPU direct read of CPU memory reaches ~415 GB/s locally, ~84 GB/s from another Grace chip
On NCSA DeltaAI (4x GH200 per node, PyTorch 2.12.0+cu130), I timed reading k random 9.6 MB blocks from a 16 GiB bf16 CPU tensor. The tensor was pinned in place with torch.cuda.cudart().cudaHostRegister and wrapped through __cuda_array_interface__, then read on the GPU with index_select. Median of 20 runs, CUDA events. Results: with the memory on the same Grace chip (srun --mem-bind=local), the direct read reached 194 GB/s at k=1 and 415 GB/s at k=64. A pinned contiguous .cuda(non_blocking=True) copy reached 418 GB/s, close to the 450 GB/s NVLink-C2C spec. With the memory on the next Grace chip (numactl --membind=<other node>), copy and direct read fell to about 84-89 GB/s. The CPU gather (index_select into a pinned buffer with 16 cores) reached only about 16 GB/s, so it is limited by the CPU, not the link. Each Grace chip has only about 120 GB of its own memory (numactl -H), so the fast region per GPU is about 120 GB even though free shows 476 GB. Bind memory with --mem-bind=local before you measure anything.