The Split Personality
Most Linux hardware drivers are monolithic — a single kernel module handles everything from interrupt processing to sending commands. The NVIDIA driver breaks this convention deliberately, splitting into two large components that serve very different roles.
1. The Kernel Mode Driver (nvidia.ko)
This is a Linux Kernel Module — a .ko file loaded into the kernel at boot or when the GPU is first accessed. It operates in Ring 0 (kernel space) with full hardware privileges. Its responsibilities are the "boring but critical" ones:
- Allocating and mapping physical GPU memory into the CPU's virtual address space.
- Handling GPU interrupts (signals the GPU sends to the CPU to say "I'm done" or "something went wrong").
- Managing power states (P-states) and clock frequencies.
- Enforcing isolation between multiple users or processes sharing the same GPU.
- Registering the GPU as a character device (
/dev/nvidia0,/dev/nvidiactl,/dev/nvidia-uvm).
2. The User Mode Driver (libcuda.so / libnvidia-ml.so)
This shared library runs in Ring 3 (user space), inside the same process as your application. It is where the actual intelligence lives:
- JIT Compilation: It takes your CUDA C++ code (compiled to PTX, the virtual ISA) and compiles it to SASS (the actual GPU machine code) for the specific GPU model at runtime.
- Command Submission: Once the kernel driver has set up memory mappings, the user-mode driver writes GPU commands directly into a command ring buffer via Memory Mapped I/O (MMIO). The GPU's hardware reads from this ring buffer continuously. This means submitting work to the GPU does not require a system call — it is a direct memory write, as fast as writing to RAM.
- Stream and Context Management: It manages CUDA streams (queues of GPU work), synchronization primitives, and the CUDA context (the GPU equivalent of a process).
The critical insight is the bypass: after initial setup by the kernel driver, the hot path for submitting GPU work goes Application → libcuda.so → MMIO write → GPU hardware, completely skipping the kernel. This eliminates the ~1–5 µs overhead of a system call for every kernel launch.
Interviewers at FAANG-scale companies often ask: "Why doesn't every GPU operation require a system call?" The answer is MMIO-based command submission. The kernel driver maps a region of GPU registers into the process's virtual address space. Writing to that memory region directly signals the GPU without entering kernel mode. This is the same mechanism used by high-performance network cards (DPDK, RDMA) to achieve kernel-bypass networking.