System Design

Load Balancing Algorithms Explained (with Code)

A load balancer is only as good as the algorithm deciding where each request goes. Here are the six algorithms that matter, each with working code, when to use it, and where it breaks.

By Admin2026-06-1814 min readload-balancingalgorithmssystem-designscalabilitydistributed-systemsbackend

A load balancer's whole job is to answer one question, over and over, millions of times: which server should handle this request? The logic it uses to answer is the load balancing algorithm, and the choice matters more than people expect. The wrong algorithm leaves some servers overwhelmed while others idle, drives up latency, and wastes capacity you're paying for. The right one keeps load even, response times low, and the whole pool humming.

This guide walks through the six load balancing algorithms you're most likely to meet, each with working code so the logic is concrete rather than abstract. We assume you already know what a load balancer is and what it's for - if not, start with our guide on how load balancers actually work, then come back here for the algorithms in depth.

Two families: static and dynamic

Before the algorithms themselves, one organizing distinction that clarifies all of them. Load balancing algorithms fall into two families:

Static algorithms decide where a request goes without looking at the current state of the servers. They follow a fixed rule - a rotation, a weight, a hash. They're simple and fast, but they can't react if a server happens to be busier than the others. Round robin, weighted round robin, and IP hash are static.
Dynamic algorithms make decisions based on the real-time state of the servers - how many connections each has, how fast each is responding. They adapt to reality, distributing more intelligently, at the cost of needing to track that state. Least connections and least response time are dynamic.

Keep this distinction in mind as we go: static algorithms are about predictable distribution; dynamic algorithms are about adapting to actual load.

1. Round Robin

The simplest algorithm of all: cycle through the servers in order. First request to server A, second to B, third to C, fourth back to A, and so on. Every server gets an equal share of requests in strict rotation.

class RoundRobin:
    def __init__(self, servers):
        self.servers = servers
        self.index = -1

    def next_server(self):
        self.index = (self.index + 1) % len(self.servers)
        return self.servers[self.index]

lb = RoundRobin(["A", "B", "C"])
print([lb.next_server() for _ in range(7)])   # A B C A B C A

When to use it: when your servers are roughly identical in capacity and your requests take roughly the same amount of work. It's the sensible default for a uniform fleet.

Where it breaks: round robin is blind to load. If one request happens to be expensive and ties up a server, round robin keeps sending it more requests anyway, because it's just counting in a circle. It also ignores differences in server power - a weak server gets the same share as a strong one.

2. Weighted Round Robin

A refinement for fleets where servers aren't identical. Each server gets a weight reflecting its capacity, and receives requests in proportion to that weight. A server with weight 3 gets three times as many requests as one with weight 1.

def build_weighted_pool(weighted_servers):
    pool = []
    for server, weight in weighted_servers.items():
        pool.extend([server] * weight)
    return pool

class WeightedRoundRobin:
    def __init__(self, weighted_servers):
        self.pool = build_weighted_pool(weighted_servers)
        self.index = -1

    def next_server(self):
        self.index = (self.index + 1) % len(self.pool)
        return self.pool[self.index]

# Server A is 3x as powerful as B
lb = WeightedRoundRobin({"A": 3, "B": 1})
picks = [lb.next_server() for _ in range(8)]
print(picks)
print("A:", picks.count("A"), " B:", picks.count("B"))   # A gets ~3x of B

When to use it: when your servers have different capacities - a common situation when you've added newer, more powerful machines to an older fleet, or run mixed instance sizes.

Where it breaks: it's still static, so like plain round robin it doesn't react to actual current load - only to the fixed weights you assigned. (Real implementations also interleave the picks more smoothly than the simple expand-the-pool version above, to avoid sending three requests to A in a row, but the proportion is the point.)

3. Least Connections

The first dynamic algorithm: send each new request to the server currently handling the fewest active connections. This adapts to reality - if one server is bogged down with slow, long-running requests, it has more open connections, so new requests are routed elsewhere.

class LeastConnections:
    def __init__(self, servers):
        self.connections = {s: 0 for s in servers}

    def assign(self):
        server = min(self.connections, key=self.connections.get)
        self.connections[server] += 1
        return server

    def release(self, server):
        self.connections[server] -= 1

lb = LeastConnections(["A", "B", "C"])
print(lb.assign())   # A (all at 0, picks first)
print(lb.assign())   # B
print(lb.assign())   # C
lb.release("A")      # A finishes its request
print(lb.assign())   # A (now has the fewest again)

When to use it: when request durations vary widely. If some requests finish in milliseconds and others take many seconds, least connections naturally steers traffic away from servers stuck on the slow ones. It's the go-to for workloads with uneven request costs.

Where it breaks: it assumes a server with fewer connections is less loaded, which isn't always true - a server might have few connections but each one extremely heavy. It also needs to track live connection counts, which is the cost of being dynamic.

4. Weighted Least Connections

Exactly what it sounds like: combine the two ideas. Account for both the current connection count and each server's capacity, so a powerful server can comfortably hold more connections before it's considered "loaded." The decision is based on connections relative to capacity - effectively, connections divided by weight.

class WeightedLeastConnections:
    def __init__(self, weighted_servers):           # {server: weight}
        self.weights = weighted_servers
        self.connections = {s: 0 for s in weighted_servers}

    def assign(self):
        # pick the server with the lowest connections-per-capacity ratio
        server = min(self.connections,
                     key=lambda s: self.connections[s] / self.weights[s])
        self.connections[server] += 1
        return server

lb = WeightedLeastConnections({"A": 3, "B": 1})   # A is 3x more capable
print([lb.assign() for _ in range(8)])   # A takes more, proportional to capacity

When to use it: the best of both worlds when you have both varied server capacities and varied request durations. This is a common default in mature production load balancers.

5. IP Hash (Hash-Based)

A different idea entirely. Instead of rotating or measuring load, hash an attribute of the request - typically the client's IP address - and use the hash to pick a server. The key property: the same client always maps to the same server (as long as the server pool doesn't change).

def ip_hash(client_ip, servers):
    return servers[hash(client_ip) % len(servers)]

servers = ["A", "B", "C"]
for ip in ["1.2.3.4", "5.6.7.8", "9.9.9.9", "1.2.3.4"]:
    print(f"{ip:12} -> {ip_hash(ip, servers)}")
# note the same IP (1.2.3.4) always lands on the same server

When to use it: when you need a client to consistently reach the same server - for example, when that server holds session state in memory (sticky sessions). The hash gives you that affinity without storing a mapping.

Where it breaks: distribution can be uneven if some clients are far more active than others, and - critically - adding or removing a server changes the modulo, remapping nearly everyone. This is exactly the problem consistent hashing solves, by remapping only a small fraction of clients when the pool changes. For dynamic pools, consistent hashing is the better hash-based choice.

6. Least Response Time

The most sophisticated of the common algorithms. It sends each request to the server with the lowest combination of active connections and current response time - favoring the server that's both least busy and fastest right now. It's dynamic and adapts continuously to how the servers are actually performing.

class LeastResponseTime:
    def __init__(self, servers):
        self.connections = {s: 0 for s in servers}
        self.response_ms = {s: 0 for s in servers}   # measured live

    def assign(self):
        # score = active connections * recent average response time
        server = min(self.connections,
                     key=lambda s: self.connections[s] * (self.response_ms[s] + 1))
        self.connections[server] += 1
        return server

lb = LeastResponseTime(["A", "B", "C"])
lb.response_ms = {"A": 200, "B": 50, "C": 50}   # A is responding slowly
print([lb.assign() for _ in range(6)])   # favors the faster B and C

When to use it: when response time is what you care about most and servers' performance varies moment to moment (because of GC pauses, varying load, or differing hardware). It routes around slow servers automatically.

Where it breaks: it needs continuous, accurate measurement of response times, which adds overhead and complexity. The extra intelligence only pays off when performance genuinely varies.

Comparison

Algorithm	Family	Considers	Best for
Round Robin	Static	Nothing (just rotates)	Identical servers, uniform requests
Weighted Round Robin	Static	Server capacity (fixed)	Servers of different power
Least Connections	Dynamic	Live connection counts	Variable request durations
Weighted Least Connections	Dynamic	Connections + capacity	Varied servers and requests
IP Hash	Static	Client identity	Session affinity / sticky sessions
Least Response Time	Dynamic	Connections + speed	Latency-sensitive, varying performance

How to choose

A practical decision path:

Identical servers, similar requests? → Round Robin. Don't overthink it.
Different server capacities? → Weighted Round Robin (or Weighted Least Connections if requests also vary).
Request durations vary a lot? → Least Connections.
Need clients pinned to the same server? → IP Hash (or consistent hashing for dynamic pools).
Latency is critical and performance fluctuates? → Least Response Time.

There's also a clever lightweight option worth knowing: "power of two choices" - pick two servers at random and send the request to whichever has fewer connections. It captures most of the benefit of least-connections distribution with far less coordination, which makes it popular at large scale.

Interview: If asked about load balancing, don't just name round robin and stop. Show range: "Round robin is fine for uniform servers, but if request durations vary I'd use least connections so slow requests don't pile up on one server. If I needed session affinity I'd use a hash-based approach - ideally consistent hashing so scaling the pool doesn't remap everyone." Naming the static-versus-dynamic distinction, and connecting IP hash to consistent hashing, signals real depth.

Algorithms work together with health checks

An important real-world point: the algorithm doesn't operate alone. It works on top of the load balancer's health checks. Before any algorithm picks a server, the load balancer has already removed unhealthy servers from consideration - so round robin rotates only among the healthy servers, least connections compares only the healthy ones, and so on. The algorithm decides distribution; the health check decides eligibility. Together they deliver both balance and resilience: traffic is spread evenly and never sent to a dead server.

This is why you sometimes see an algorithm "skip" a server - it's not the algorithm's logic, it's that the server failed its health check and was temporarily removed from the pool. When it recovers and passes checks again, the algorithm starts including it once more.

A worked scenario: same traffic, different algorithms

To see why the choice matters, imagine three servers and a burst of requests where one request is very slow - it ties up a server for a long time - while the rest are quick.

Under round robin, the slow request lands on, say, server B. Round robin keeps rotating regardless, so it cheerfully sends server B its share of new requests even though B is stuck on the slow one. Those new requests queue up behind the slow request, and users hitting server B experience delays - while servers A and C may have spare capacity. Round robin's blindness to load hurts here.

Under least connections, the picture is different. The slow request keeps server B's connection count high (the connection stays open while the slow work runs), so least connections stops sending B new requests and routes them to A and C instead. The slow request is contained to B, and everyone else is served quickly. The dynamic algorithm adapted; the static one didn't.

This is the static-versus-dynamic distinction made concrete: when request costs are uniform, the simple static algorithm is fine; when they vary, a dynamic algorithm that watches real load prevents one heavy request from dragging down everyone routed behind it.

Where the algorithm runs: Layer 4 vs Layer 7

The algorithm's options depend partly on what information the load balancer can see. A Layer 4 (transport-level) load balancer routes on IP and port without inspecting request content, so it works well with algorithms like round robin, least connections, and IP hash that don't need to understand the request. A Layer 7 (application-level) load balancer can read the HTTP request - its path, headers, and cookies - which enables content-aware routing (sending /video/* to media servers and /api/* to API servers) on top of the basic distribution algorithm. So the layer the load balancer operates at shapes which strategies are even available, with Layer 7 offering the richer set at a slightly higher processing cost.

Algorithm and session affinity together

One more subtlety: session affinity (keeping a client on the same server) can be layered on top of a distribution algorithm. A load balancer might use least connections to initially place a client, then "remember" that placement so the client returns to the same server. But the better long-term approach is usually to make servers stateless - storing session state externally - so you can use a pure distribution algorithm without needing affinity at all. Affinity is a workaround for state living in the wrong place; the algorithm shines brightest when servers are interchangeable.

The bottom line

The load balancing algorithm is the brain of the load balancer, and the right choice depends on your servers and your traffic. Static algorithms (round robin, weighted round robin, IP hash) are simple and predictable but blind to real load. Dynamic algorithms (least connections, least response time) adapt to what's actually happening, at the cost of tracking server state. Match the algorithm to your situation - uniform or varied servers, uniform or varied requests, whether you need session affinity - and you keep the whole pool balanced and fast.

These algorithms are a core part of how systems scale horizontally, and we work through them with real systems across System Design Fundamentals.

Go deeper

Load balancing (Wikipedia) - the algorithms and architectures in depth.
Round-robin scheduling (Wikipedia) - the simplest distribution strategy.
Weighted round robin (Wikipedia) - handling servers of differing capacity.
What is load balancing? (Cloudflare) - a clear, practical overview of the algorithms in use.