The Fast Path: GPUDirect Storage (GDS)

Bypassing the CPU

To understand GPUDirect Storage, you first need to understand the traditional data path — and why it wastes resources.

The Traditional Path (with "Bounce Buffer"):

Disk/NIC → System RAM (CPU copies here) → GPU VRAM
              ↑
         CPU is involved in every byte transfer

1. The storage controller reads data from disk into a buffer in system RAM.
2. The CPU copies that buffer from system RAM into GPU VRAM (via PCIe).
3. The GPU can now process the data.

This path has two problems. First, it consumes CPU cycles — the CPU is acting as a data mover, not doing useful computation. Second, every byte of data crosses the PCIe bus twice: once from the storage controller to RAM, and once from RAM to the GPU. PCIe bandwidth is finite and shared.

GPUDirect Storage (GDS):

GDS, introduced by NVIDIA, allows the NVMe controller or network card (RDMA-capable NIC) to write data directly into GPU VRAM using DMA (Direct Memory Access), bypassing system RAM entirely.

NVMe/NIC → GPU VRAM (direct, via DMA)
              ↑
         CPU not involved in data movement

The result:
- Zero-copy: Data moves once across PCIe, not twice.
- CPU freed: CPU cycles are available for preprocessing, decoding, augmentation.
- Lower latency: No CPU scheduling overhead in the data path.

NVIDIA's kvikio Python library exposes this through a CuFile object that reads directly into a GPU buffer (allocated with CuPy), rather than into a CPU-side NumPy array first — the read call is the only thing that changes in application code, but it eliminates the CPU-RAM hop entirely.

GDS requires: an NVIDIA GPU (Volta or later), a compatible NVMe drive or RDMA NIC, and the nvidia-fs kernel module. It is most impactful for workloads where the CPU is the bottleneck in the data pipeline — genomics (large binary files), 4K/8K video training, and scientific simulations with large tensor checkpoints.