Why TCP/IP Is Too Slow for Supercomputing
Consider what happens when you send a network packet using standard TCP/IP on Linux. Your application calls a socket API. The OS kernel takes control (a context switch from user space to kernel space), processes the packet through the TCP/IP stack, hands it to the NIC driver, and the NIC transmits it. On the receiving end, the NIC interrupts the CPU, the kernel processes the incoming packet through the TCP/IP stack, and finally delivers it to the application. This round trip through the kernel takes on the order of 50–100 microseconds of CPU time per message.
For a web server handling HTTP requests, this overhead is negligible. For a distributed ML training job where 1,000 GPUs need to synchronize gradient updates thousands of times per second, it is catastrophic. The GPUs spend more time waiting for network synchronization than doing actual computation.
There is also a structural problem with TCP called Head-of-Line (HOL) Blocking: TCP guarantees in-order delivery. If packet 50 in a stream is lost, packets 51 through 100 must wait in a buffer until packet 50 is retransmitted and received — even if those later packets have already arrived. In a high-speed cluster with thousands of simultaneous flows, this creates cascading stalls.
EFA and the SRD Protocol
Elastic Fabric Adapter (EFA) is a specialized Network Interface Card (NIC) available on specific EC2 instance types (such as p4d.24xlarge, p5.48xlarge, c5n.18xlarge, and hpc6a.48xlarge). It solves the kernel overhead and HOL blocking problems through two mechanisms:
1. OS Bypass via libfabric
EFA exposes a user-space API through the libfabric library (part of the OpenFabrics Interfaces standard). Applications — specifically MPI (Message Passing Interface) runtimes and NCCL (NVIDIA Collective Communications Library) — use libfabric to send and receive messages directly to/from the EFA hardware, completely bypassing the OS kernel. No context switches. No kernel TCP/IP stack traversal. Latency drops from ~100 microseconds to single-digit microseconds.
2. Scalable Reliable Datagram (SRD)
Instead of TCP, EFA uses SRD, a protocol developed by AWS specifically for this use case. SRD's key properties:
- Multi-path transmission: SRD sprays packets for a single message across multiple network paths simultaneously using ECMP (Equal-Cost Multi-Path) routing. This exploits the full bisection bandwidth of the underlying network fabric.
- Out-of-order delivery: Unlike TCP, SRD does not require packets to arrive in order. The receiving EFA hardware reassembles them correctly regardless of arrival order. This eliminates HOL blocking entirely.
- Fast loss recovery: SRD detects and recovers from packet loss significantly faster than TCP's retransmission timeout mechanisms, using selective acknowledgment at the hardware level.
The combination of OS bypass and SRD means that EFA-equipped instances can achieve near-wire-speed communication with latency comparable to InfiniBand — the technology used in traditional on-premises supercomputers.
What EFA Is and Is Not For
EFA is designed exclusively for tightly-coupled, instance-to-instance communication within a cluster. Specifically:
- ML training: NCCL AllReduce operations between GPU nodes during distributed training (e.g., training a large language model across 512 GPUs).
- HPC simulations: MPI-based workloads like computational fluid dynamics, weather modeling, or finite element analysis where nodes exchange boundary condition data every iteration.
- Financial modeling: Monte Carlo simulations that partition work across nodes and aggregate results.
EFA cannot communicate with the public internet, S3, or other standard AWS services. It operates only within the cluster's placement group. For all other network traffic — downloading datasets from S3, sending results to a database, API calls — the instance uses a standard Elastic Network Interface (ENI) running normal TCP/IP. A single EC2 instance can have both an EFA and one or more ENIs attached simultaneously.
A common interview question is: "How would you network a 1,000-GPU training cluster on AWS?" The expected answer covers three layers: (1) EFA for inter-node gradient synchronization via NCCL, (2) placement in a Cluster Placement Group to ensure instances are on the same high-bandwidth network spine, and (3) standard ENIs for S3 dataset access. Candidates who only mention EFA without mentioning Cluster Placement Groups miss a critical point — EFA's benefits are maximized when instances are physically co-located on the same network fabric.
Key Point — EFA requires application-level support. You cannot simply attach an EFA to an existing EC2 instance and get faster networking automatically. The application must be written to use MPI or NCCL, which in turn use the libfabric API to issue EFA-native operations. A standard web server or database gains nothing from EFA. This is a purpose-built tool for a specific class of scientific and ML workloads.