Free lesson · System Design Fundamentals

Long Polling

Easy6 min readsystem-designhlddistributed-systems

Lesson 8.1 - Long Polling

The first lesson of Module 8.

Before WebSockets existed, before Server-Sent Events were standardized, engineers needed a way for servers to push data to browsers. The simplest approach - short polling - is the client asking "anything new?" every few seconds. It works, but it's wasteful: 99% of the time the server responds "nope, nothing new," and the request was pointless. Multiply by millions of clients polling every 2 seconds, and you have a staggering amount of wasted bandwidth, server CPU, and battery life.

Long polling is the clever improvement: the client sends a request, and the server holds the connection open until it has something to send - or until a timeout expires. Instead of "ask every 2 seconds and usually get nothing," it's "ask once and wait until there's actually something." When data arrives, the server responds immediately. The client processes the response and immediately opens a new long-polling request. The result: near-real-time delivery with far less wasted traffic than short polling.

Interactive animation: The wasteful baseline: short polling asks again and again, and most answers are empty.

How It Works

sequenceDiagram
    participant C as Client
    participant S as Server
    
    Note over C,S: SHORT POLLING (wasteful)
    C->>S: Any updates? (t=0s)
    S-->>C: No (empty response)
    C->>S: Any updates? (t=2s)
    S-->>C: No (empty response)
    C->>S: Any updates? (t=4s)
    S-->>C: No (empty response)
    C->>S: Any updates? (t=6s)
    S-->>C: Yes! New message: "Hello"
    
    Note over C,S: 4 requests, 3 were wasted
    
    Note over C,S: LONG POLLING (efficient)
    C->>S: Any updates? (t=0s)
    Note over S: Hold connection open...\nwaiting for data...
    Note over S: 6 seconds pass...\nnew message arrives!
    S-->>C: Yes! New message: "Hello" (t=6s)
    C->>S: Any updates? (t=6s, immediately reconnect)
    Note over S: Hold connection open...\nwaiting...

A side-by-side comparison of short polling and long polling. With short polling, the client sends a request every 2 seconds asking "any updates?" At t=0, t=2, and t=4, the server responds "no" - three wasted round trips consuming bandwidth, server CPU, and client battery. At t=6, the server finally has data and responds "yes, new message: Hello." Four requests total, three completely wasted. With long polling, the client sends one request at t=0. The server receives it but doesn't respond immediately - it holds the connection open, waiting for data to arrive. Six seconds pass. A new message arrives at the server. The server immediately responds with the message. The client processes the response and immediately opens a new long-polling request. One request, zero waste. The data arrived with the same latency (6 seconds) but used 75% fewer requests. At scale - millions of clients, thousands of messages per second - this reduction is enormous. Long polling isn't truly real-time (there's a brief gap during reconnection), but it's near-real-time with dramatically less overhead than short polling.

The long polling lifecycle:

Client sends request. A standard HTTP request - GET /api/events?since=last_event_id.
Server holds the connection. Instead of responding immediately, the server keeps the connection open. It checks for new data. If there's nothing new, it waits - typically up to 30-60 seconds (the long-poll timeout).
Data arrives → immediate response. When new data becomes available (a chat message, a notification, a status change), the server responds immediately with the data. Latency from event occurrence to client delivery: milliseconds (the connection is already open).
Timeout → empty response. If the timeout expires without new data, the server responds with an empty result (or a "no data" status). This prevents the connection from being held indefinitely and allows intermediaries (load balancers, proxies) that have their own timeouts to not kill the connection.
Client reconnects immediately. Whether the response contained data or was a timeout, the client immediately sends a new long-polling request. The cycle continues.

Interactive animation: The server holds the request open until there is news, answers, and the client asks again at once.

When Long Polling Is the Right Answer

Long polling has been largely superseded by WebSockets and SSE for most real-time use cases. But it still has legitimate applications:

1. Environments where WebSocket is blocked. Some corporate firewalls, proxies, and network configurations block WebSocket connections (they look like "suspicious" long-lived TCP connections). Long polling uses standard HTTP requests - nothing to block, nothing to configure. If your users are behind restrictive corporate networks, long polling might be the only option.

2. Simple notification systems. A dashboard that updates every few seconds when new data arrives. The data volume is low (a few events per minute), bidirectional communication isn't needed (server pushes to client, client doesn't push back), and the infrastructure complexity of WebSocket or SSE isn't justified.

3. Fallback mechanism. Many real-time libraries (Socket.IO, SignalR) use WebSocket as the primary transport and fall back to long polling when WebSocket is unavailable. Long polling is the universal fallback - it works everywhere HTTP works.

4. Serverless environments. AWS Lambda, Cloud Functions, and other serverless platforms don't support persistent connections (WebSocket, SSE). Long polling works because each request is a standard HTTP invocation - the function executes, holds for data (or times out), responds, and terminates. The client reconnects with a new function invocation.

The Downsides

1. Connection overhead. Each long-poll response requires the client to establish a new HTTP connection (unless using HTTP/2 with persistent connections). The TCP handshake + TLS handshake adds latency on each reconnection - 50-200ms depending on network conditions.

2. Server resource consumption. Each waiting long-poll request holds a server connection open. With 100,000 concurrent clients, that's 100,000 open connections on your servers - consuming memory for each connection's state. This is manageable with async/event-driven servers (Node.js, Nginx) but problematic with thread-per-connection servers (traditional Java Servlets, PHP).

3. Not truly bidirectional. Long polling only delivers data server-to-client. For the client to send data to the server, it uses a separate regular HTTP request. This is fine for notification patterns but insufficient for real-time collaboration or chat where both sides send frequently.

4. Latency gap during reconnection. Between receiving a response and establishing the next long-poll request, there's a brief gap (50-200ms). Events that occur during this gap are delivered on the next request, not instantly. For most applications this is unnoticeable; for sub-10ms latency requirements, it's not acceptable.

A Brandon Story: The Notification System That Scaled

Interactive animation: 50,000 users short polling, then one held request each.

A team I'd advised built a notification bell for their web dashboard - the orange dot that appears when you have new notifications. Their first implementation: short polling every 3 seconds. With 50,000 concurrent users, that was 16,667 requests per second - 99% of which returned "no new notifications." Their API servers were spending more CPU on notification polling than on actual business logic.

They switched to long polling. Each client sent one request; the server held it open for up to 30 seconds. When a notification arrived, the server responded immediately to the relevant user's held connection. When 30 seconds passed without a notification, the server responded with "no data" and the client reconnected.

Results:

Requests per second: 16,667 → ~1,700 (most connections held for the full 30-second timeout: 50,000/30 ≈ 1,667 reconnections per second). A 90% reduction in request volume.
P95 notification delivery latency: ~3 seconds (short polling, waiting for the next poll) → ~50ms (long polling, connection already open). 60x improvement.
Server CPU for notification handling: reduced by 85%.

They later evaluated WebSocket but decided long polling was sufficient - the notification bell needed server-to-client push (not bidirectional), the update frequency was low (a few notifications per user per hour), and long polling worked through every corporate firewall their enterprise customers used. Sometimes the older, simpler technique is the right one.

Lessons:

Short polling at scale wastes enormous resources - 99% of responses are empty
Long polling reduces request volume by 90%+ and improves delivery latency from seconds to milliseconds
For low-frequency, server-to-client notification patterns, long polling is genuinely sufficient
The "newest" technology (WebSocket) isn't always the right choice - evaluate the actual requirements
Corporate firewall compatibility is a real-world constraint that favors long polling over WebSocket

Six Takeaways from This Lesson

Short polling = ask every N seconds. Wasteful - most responses are empty. Doesn't scale.
Long polling = ask once, server holds connection until data arrives (or timeout). Near-real-time with 90%+ less traffic than short polling.
The lifecycle: client requests → server holds → data arrives (instant response) or timeout (empty response) → client reconnects immediately.
Still useful for: restricted networks (firewalls block WebSocket), serverless environments (no persistent connections), simple notification systems, and WebSocket fallback.
Downsides: reconnection latency gap, connection overhead per reconnect, not bidirectional, server holds many open connections.
Long polling was sufficient for notifications at 50K concurrent users - 90% fewer requests, 60x lower latency than short polling. Don't reach for WebSocket until you need bidirectional or sub-10ms delivery.

What's Next

Lesson 8.2 covers WebSockets - persistent bidirectional connections that solve long polling's limitations for chat, gaming, collaboration, and any use case where both client and server need to send data frequently.

Back to System Design Fundamentals