Not All Requests Are Equal
The simplest load balancing algorithm is Round Robin: send the first request to Server 1, the second to Server 2, the third to Server 3, then back to Server 1. It's fair in the sense that each server gets the same number of requests — but it assumes all requests take the same amount of time and all servers have the same capacity. Neither assumption is usually true.
Weighted Round Robin addresses the capacity problem. If Server A has 32 CPU cores and Server B has 8, you assign Server A a weight of 4 and Server B a weight of 1. The LB sends 4 requests to A for every 1 it sends to B. This is configured explicitly in the LB and requires you to know your server capacities in advance.
Least Connections addresses the variable request duration problem. Consider:
- Request A (fetch a thumbnail): completes in 5ms.
- Request B (generate a PDF report): takes 5,000ms.
With Round Robin alternating 1-2-1-2, Server 2 might accumulate 50 long-running PDF jobs while Server 1 breezes through 50 thumbnail requests. Server 2 crashes; Server 1 is idle.
With Least Connections, the LB tracks the number of active open connections to each backend at all times:
- Server A: 50 active connections
- Server B: 3 active connections
- New request arrives → goes to Server B
This naturally routes new work to the server with the most available capacity, regardless of how many total requests it has handled. It is the right default for workloads with variable request durations (API servers, microservices, anything with mixed fast and slow endpoints).
Least Response Time is a further refinement used by some LBs (like HAProxy with the leastconn + response time tracking, or Nginx Plus): it routes to the backend with the lowest combination of active connections and average response time. This is the most accurate reflection of actual backend health but requires the LB to actively measure response times.