AlgoInsight

System Design

Design WhatsApp

A messaging system holding hundreds of millions of live connections, delivering messages in milliseconds, knowing who's online - and never losing a message. It's the purest test of real-time-at-scale I know. Here's how to design it.

By Admin2026-06-1816 min readwhatsappchat-systemsystem-designcase-studyinterviewreal-time

"Design WhatsApp" - or any chat system: real-time messaging between users, one-to-one and in groups - is the question I reach for when I want to see whether a candidate truly understands real-time systems at scale. There's nowhere to fake it. A chat system has to hold an enormous number of live connections open at once, route a message from any sender to any recipient in milliseconds, track who's online, and never lose a message even when people are offline. Each of those is a real problem, and together they make one of the most revealing case studies in the interview canon.

Let me walk through how I'd want to see this built - and, as the person scoring it, exactly where the design is won.

Scope the core

Narrow first, as always. The functional core of a chat system:

The non-functional requirements that shape everything:

Interview: The framing I want to hear immediately is that this is a persistent-connection problem at heart. A chat system isn't request-response - clients stay connected so messages can be pushed to them the instant they arrive. Candidates who recognize early that the central challenge is managing a vast number of long-lived connections, and routing messages between them, have grasped what makes this hard. Candidates who design it like a REST API with polling have missed the entire point.

The foundation: persistent connections

For messages to arrive instantly, the server must be able to push to clients - which means each client maintains a persistent connection to the server, typically a WebSocket. When you send a message, it travels over your connection to the server, and the server pushes it down the recipient's open connection. No polling; real-time push in both directions. (See the WebSockets guide for how this works; the key design point is that polling can't deliver instant messaging, and persistent connections can.)

This immediately raises the defining challenge of the whole system: a chat service with hundreds of millions of users must hold hundreds of millions of persistent connections open simultaneously. No single server can do that. So we need many connection servers (sometimes called gateway or chat servers), each holding a share of the connections, with users distributed across them.

graph TD
    A[User A] -->|WebSocket| CS1[Connection Server 1]
    B[User B] -->|WebSocket| CS2[Connection Server 2]
    CS1 --> ROUTE[Message routing layer<br/>pub-sub / message queue]
    CS2 --> ROUTE
    ROUTE --> CS1
    ROUTE --> CS2
    CS1 --> STORE[(Message store)]
    CS2 --> STORE

This diagram shows a chat system's core. User A connects via WebSocket to Connection Server 1, and User B to Connection Server 2 - different servers, because no single server can hold all connections. A message routing layer, built on pub-sub or a message queue, sits between the connection servers, routing messages from one server to another. Messages are also written to a message store for persistence and history.

The defining challenge: routing between servers

Here's the problem that makes this design genuinely hard, and the part I'm most interested in. User A is connected to connection server 1. User B is connected to connection server 7. When A sends a message to B, server 1 receives it - but B isn't connected to server 1, so server 1 can't just push it down a local connection. How does the message get from server 1 to server 7?

This is the heart of the design. The system needs a way to know which connection server a given user is connected to, and to route a message to that server. The standard approaches:

The candidate who recognizes that the cross-server routing problem is the central challenge - and proposes a registry plus an internal messaging backbone to solve it - is demonstrating exactly the systems thinking this question is designed to surface.

Interview: This routing problem is where I separate good from great. A candidate who designs a single chat server and never confronts "what happens when the two users are on different servers" has skipped the actual hard part. A candidate who says "I'll track which server each user is connected to, and route messages between connection servers through an internal pub-sub layer" has identified and solved the core challenge. That move - seeing that the difficulty is between the servers, not within one - is the strongest signal in this entire interview.

Reliability: the offline problem

A message must not be lost if the recipient is offline when it's sent. So the system can't simply "push it to their connection" - there might be no connection. The design needs to handle this: when a recipient is offline, the message is stored (queued) and delivered when they reconnect. When B comes back online, the server checks for any messages that arrived while they were away and pushes them.

This is why messages are persisted, not just relayed. Every message is written to a message store, both for offline delivery and for message history. The delivery status - sent, delivered, read - is tracked through this flow: "sent" when the server receives it, "delivered" when it's pushed to the recipient's connected device, "read" when the recipient's app confirms they've seen it. Modeling this status flow correctly, and handling the offline case via stored-and-forwarded messages, is a core part of a complete answer.

Presence: who's online

The "online / last seen" feature has a subtle scale challenge. Presence changes constantly as people connect and disconnect, and naively broadcasting every status change to everyone who might care would be enormous load. The system tracks each user's connection status (a user is "online" while they hold an active connection, detected via the connection itself and heartbeats), and surfaces presence efficiently - typically updating it for a user's contacts rather than broadcasting globally, and often tolerating slight staleness (last-seen being a few seconds old is fine). A candidate who notes that presence at scale requires care - not naively notifying everyone of every status change - is thinking about the real costs.

Group messaging

Group chat adds fan-out: a message to a group of 100 people must reach all 100. The server takes the group message and delivers it to each member - pushing to those currently connected, storing for those offline. For small-to-moderate groups this is straightforward fan-out. For very large groups, the fan-out cost grows, and the design must handle delivering one message to many recipients efficiently - the same fan-out consideration that appears in feed systems, applied to group delivery.

Deep dive: estimating the scale

A quick estimate frames why this is hard. Say hundreds of millions of daily active users, with tens of billions of messages sent per day - that's on the order of hundreds of thousands to millions of messages per second at peak. But the more striking number is connections: if even a few hundred million users are connected at once, that's a few hundred million persistent connections to hold open simultaneously. A single server can handle perhaps tens of thousands to low hundreds of thousands of connections, so you need thousands of connection servers just to hold all the open sockets. That arithmetic is what forces the sharded-connection-server architecture and makes the cross-server routing problem unavoidable. Stating these numbers shows the interviewer you understand why a single-server design is a non-starter.

Deep dive: end-to-end encryption

WhatsApp is known for end-to-end encryption, and an interviewer may probe it. The key design implication is that messages are encrypted on the sender's device and only decrypted on the recipient's - the server relays encrypted payloads it cannot read. This affects the design specifically: the server can still route and store messages (it's moving opaque encrypted blobs), but it cannot do anything that requires reading message content (like server-side search of message text). You don't need to design the cryptography, but recognizing that end-to-end encryption means the server handles messages as opaque payloads - routing and storing without reading - shows you understand its architectural consequence. It's a good example of a requirement that constrains the design without changing its core shape.

Deep dive: multi-device and message sync

Modern chat isn't single-device - a user has a phone, a laptop, a tablet, and messages must sync across all of them. This adds a wrinkle: a message to a user must reach (or be available to) all their devices, and read status must sync (reading on the phone marks it read on the laptop). The design handles this by treating a user as having multiple connected devices, delivering to each, and keeping per-user message state that all devices sync against. The message store becomes the source of truth any device syncs from when it connects. A candidate who notes that multi-device support means delivering to and syncing across several connections per user - not just one - is thinking about how the system actually works today.

What I'd push a senior candidate on

For senior candidates, I probe the harder operational realities. How do you handle a connection server failing - where do its users' connections go (they reconnect, ideally to another server, and the registry updates)? How do you keep the user-to-server registry consistent at this scale and update speed (it changes every time anyone connects or disconnects)? How do you guarantee message ordering within a conversation (messages should appear in order, which requires care when they flow through async routing)? How do you store trillions of messages over time (sharding, and tiering old messages to cheaper storage)?

I also like to ask about typing indicators and presence updates that flow constantly alongside messages - these are high-frequency, low-value ephemeral signals, and a senior candidate recognizes they should be handled lightly (not persisted, tolerant of loss, sent over the connection but not through the durable message path), because treating every "user is typing" event with the rigor of a real message would waste enormous resources. Knowing which data deserves reliability guarantees and which doesn't is a mark of real systems judgment.

Deep dive: large groups and broadcast

Group messaging deserves a second look at scale, because very large groups change the calculus. A message to a small group is simple fan-out - deliver to each member. But a group with thousands of members, or a broadcast channel with millions of followers, is essentially the same fan-out-versus-celebrity problem that appears in feed systems: pushing one message to millions of recipients is expensive. The design adapts the same way - for normal groups, fan out the message to each member's delivery path; for very large groups or broadcast channels, you may not eagerly push to every member but instead let members pull recent messages when they open the chat, or use a tiered approach. A candidate who connects large-group fan-out to the broader fan-out problem - recognizing that a message to millions of recipients can't be handled the same way as a message to five - is showing they see the patterns that recur across system design, rather than treating each problem as brand new.

Putting it together

The complete architecture:

Challenge The trap The strong answer
Instant delivery Polling Persistent WebSocket connections
Hundreds of millions of connections One server Many sharded connection servers
Sender and recipient on different servers Ignore it Registry + internal pub-sub routing
Recipient offline Lose the message Store and forward on reconnect
Presence at scale Broadcast every change Efficient, contact-scoped, slightly stale

Interview: What earns a hire on "Design WhatsApp": framing it as a persistent-connection problem, recognizing that connections must be sharded across many servers, and - above all - solving the cross-server routing problem with a registry and an internal messaging backbone. Add the offline-message handling and efficient presence, and you've covered the real challenges. The candidates who struggle design a single server and never confront scale, or use polling and never achieve real-time. Nail the connection-and-routing architecture, and you've nailed the question.

The bottom line

Designing WhatsApp is fundamentally about real-time delivery at massive scale. Messages arrive instantly because clients hold persistent WebSocket connections that the server pushes to. Because hundreds of millions of connections can't live on one server, connections are sharded across many connection servers - which creates the defining challenge: routing a message between the servers when sender and recipient are on different ones, solved with a user-to-server registry and an internal pub-sub backbone. Reliability comes from persisting messages and delivering offline ones on reconnect, and presence is handled efficiently rather than broadcast.

See the connection-and-routing architecture clearly, and the rest follows - which is exactly why this question is such a clean test of whether you understand real-time systems at scale. We work through case studies like this end to end across System Design Fundamentals.

Go deeper

More articles