Free lesson · System Design Fundamentals

TCP vs UDP

Easy8 min readsystem-designhlddistributed-systems

Lesson 3.3 - TCP vs UDP

The third lesson of Module 3.

At Layer 4 (Transport), two protocols carry virtually all internet traffic: TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). TCP guarantees every byte arrives, in order, verified - like registered mail with tracking and signature confirmation. UDP guarantees nothing - like tossing a postcard into a mailbox and hoping for the best. Neither is "better"; they optimize for different properties. TCP optimizes for correctness (every byte arrives). UDP optimizes for speed (bytes go out as fast as possible, and if some are lost, that's acceptable). Understanding when each is right - and why - is essential for system design because the transport protocol determines the communication model for every service interaction.

TCP: Reliable, Ordered, Verified

TCP establishes a connection before any data flows - the famous three-way handshake:

sequenceDiagram
    participant C as Client
    participant S as Server
    
    Note over C,S: Three-Way Handshake (1 round trip)
    C->>S: SYN - "I want to connect"
    S-->>C: SYN-ACK - "OK, I'm ready"
    C->>S: ACK - "Connection established"
    
    Note over C,S: Data Transfer (reliable)
    C->>S: Data segment 1 (seq=1)
    S-->>C: ACK 1 - "received segment 1"
    C->>S: Data segment 2 (seq=2)
    Note over S: Segment 2 lost in transit!
    S--xC: (no ACK received)
    Note over C: Timeout - retransmit
    C->>S: Data segment 2 (retransmit)
    S-->>C: ACK 2 - "received segment 2"
    
    Note over C,S: Connection Teardown
    C->>S: FIN - "I'm done"
    S-->>C: FIN-ACK - "OK, me too"

A sequence diagram showing TCP's three phases. Phase one is the three-way handshake, which takes one network round trip: the client sends a SYN packet saying "I want to connect." The server responds with SYN-ACK saying "acknowledged, I'm ready." The client sends ACK confirming the connection is open. This handshake happens before any data flows, adding latency to the start of every TCP connection. Phase two is data transfer. The client sends data segment 1 with a sequence number. The server receives it and sends an acknowledgment. The client sends data segment 2. This time, the segment is lost in transit - the network dropped it. The server never receives it, so it never sends an ACK. The client waits for an ACK, hits a timeout, and retransmits segment 2. This time the server receives it and acknowledges. This retry mechanism ensures every byte is delivered, even when the network drops packets. Phase three is connection teardown - the client sends FIN to close, the server acknowledges. TCP's guarantees come at a cost: the handshake adds one round trip of latency before data flows (on a cross-continent connection, that's 150 milliseconds of waiting before the first byte is sent). The acknowledgment-and-retransmit mechanism adds overhead on every segment. And if a packet is lost, subsequent data is held back until the retransmit arrives, which is called head-of-line blocking.

What TCP guarantees:

• Delivery: every byte the sender sends, the receiver receives. Lost segments are detected (via missing ACKs) and retransmitted.
• Ordering: bytes arrive in the order they were sent. If segment 3 arrives before segment 2, TCP holds segment 3 until segment 2 arrives, then delivers both in order.
• Integrity: each segment includes a checksum (M3.6). Corrupt segments are discarded and retransmitted.
• Flow control: the receiver tells the sender how much data it can accept (via the window size), preventing the sender from overwhelming a slow receiver.
• Congestion control: the sender monitors network conditions and adjusts sending rate to avoid overwhelming the network. Algorithms include slow start, congestion avoidance, and fast retransmit.

The costs of TCP:

• Handshake latency: one round trip before any data flows. On a 150ms cross-continent link, that's 150ms of pure overhead. For short-lived connections (a single DNS lookup), the handshake might take longer than the actual data exchange.
• Head-of-line blocking: if one packet is lost, all subsequent packets (even undamaged ones) are held until the lost packet is retransmitted. This can cause latency spikes that affect the entire stream.
• Overhead per segment: sequence numbers, acknowledgments, checksums, and window management add bytes and processing to every segment.
• Not optimal for real-time: the retransmission delay for lost packets makes TCP unsuitable for applications where stale data is better than late data (live video, gaming, voice calls).

UDP: Fast, Simple, Best-Effort

UDP has no handshake, no acknowledgments, no retransmission, no ordering. The sender constructs a datagram (packet), sends it, and moves on. If it arrives - great. If it doesn't - the sender never knows. If datagrams arrive out of order - the application deals with it.

What UDP provides:

• Speed - no handshake delay, no ACK overhead, no retransmission wait
• Simplicity - minimal protocol overhead (only 8 bytes of header vs TCP's 20+ bytes)
• No head-of-line blocking - lost packets don't hold up subsequent packets
• Broadcast/multicast support - UDP can send to multiple recipients simultaneously

What UDP does NOT provide:

• No delivery guarantee - packets can be silently lost
• No ordering - packets can arrive in any order
• No duplicate detection - the same packet might arrive twice
• No congestion control - the sender can flood the network without knowing or caring

When Each Is Right

The decision depends on a simple question: what's the consequence of a lost packet?

The pattern: if correctness matters more than speed (web, APIs, files, databases), use TCP. If speed matters more than perfection and the application can tolerate some loss (video, gaming, voice, DNS, metrics), use UDP.

The nuance: many modern UDP applications build selective reliability on top - implementing only the guarantees they need, without the full TCP overhead. QUIC (the protocol underneath HTTP/3) is the prime example: it runs over UDP but provides reliable delivery, ordering per-stream (not per-connection, avoiding head-of-line blocking), encryption (TLS 1.3 built in), and congestion control. QUIC is essentially "custom TCP built on UDP with specific improvements." This pattern - UDP as a foundation with application-level reliability - is increasingly common in high-performance systems.

TCP Connection Overhead: Why It Matters for System Design

TCP's three-way handshake adds one round trip before data flows. For a single request on an established connection, this doesn't matter (the connection stays open for many requests). But for new connections, the overhead is significant:

• Same datacenter: handshake adds ~0.5ms. Negligible.
• Same continent (e.g., US East to US West): handshake adds ~30-50ms. Noticeable.
• Cross-continent (e.g., US to Europe): handshake adds ~100-150ms. Significant.
• With TLS (HTTPS): the TLS handshake adds 1-2 additional round trips on top of TCP. Total overhead for a new HTTPS connection cross-continent: 300-450ms before the first byte of data.

This is why HTTP/2 (and gRPC, which uses HTTP/2) multiplexes many requests over a single TCP connection - you pay the handshake cost once, then send thousands of requests over the same connection. It's also why connection pooling is critical for database access - establishing a new TCP connection to PostgreSQL for every query would add milliseconds of handshake overhead per query. A connection pool maintains persistent connections, amortizing the handshake cost across thousands of queries.

And it's why HTTP/3 uses QUIC over UDP - QUIC combines the TLS handshake with the connection setup into a single round trip (0-RTT for repeat connections), eliminating the separate TCP and TLS handshake overhead entirely.

A Brandon Story: The Metrics That Vanished

A team I'd advised sent application metrics to their aggregator using UDP - the standard pattern for high-volume telemetry (StatsD, the popular metrics daemon, uses UDP by default). During normal operation, packet loss was negligible (<0.01%). Dashboards looked clean. Everyone was happy.

During a network congestion event caused by a misconfigured batch job flooding the internal network, UDP packet loss spiked to 15%. Fifteen percent of metric packets never reached the aggregator. The dashboards showed gaps - CPU utilization data missing for 3-minute windows, request count graphs with suspicious dips.

For operational dashboards, this was annoying but survivable - the trends were still visible, and the gaps made it obvious something was wrong. But the team also used the same metrics pipeline for customer billing. "Requests processed" (a metric) determined what each customer was charged. The 15% packet loss meant 15% under-billing during the congestion event - $8,700 in unbilled usage for that 4-hour window.

The fix:

1. Billing-critical metrics switched to TCP - a separate pipeline with reliable delivery (Kafka, M33). If a billing metric is lost, Kafka retries. If Kafka is unavailable, the metric is buffered locally and sent when Kafka recovers.
2. Operational metrics stayed on UDP - volume matters more than perfection for dashboards. The 0.01% normal-case loss is acceptable.
3. The network congestion issue was fixed (rate limiting on the batch job), reducing packet loss back to negligible levels for both pipelines.

Lessons:

• UDP is correct for high-volume telemetry where occasional loss is acceptable (dashboards, monitoring).
• UDP is WRONG for data that has financial or contractual consequences (billing, SLA reporting, audit logging).
• The same data can have different reliability requirements depending on its use - "request count" for a dashboard (UDP fine) vs "request count" for billing (TCP required).
• Separate the pipelines. Don't run billing data through a best-effort channel.
• Test under network stress, not just normal conditions. 0.01% loss under normal operation doesn't predict 15% loss under congestion.

Six Takeaways from This Lesson

1. TCP = reliable, ordered, verified. Handshake + ACKs + retransmission. Use for correctness-critical data: web, APIs, files, databases.

2. UDP = fast, simple, best-effort. No handshake, no ACKs, no retransmission. Use for speed-critical, loss-tolerant data: video, gaming, voice, DNS, metrics.

3. TCP's handshake adds 1 round trip (+ 1-2 more for TLS). This is why connection pooling and HTTP/2 multiplexing exist - amortize the handshake cost.

4. QUIC (HTTP/3) = UDP + selective reliability. The best of both - reliable delivery without TCP's head-of-line blocking. The future of web transport.

5. The consequence of loss determines the protocol. A lost video frame is invisible. A lost payment record is a financial error. Match the protocol to the consequence.

6. Same data, different uses, different protocols. "Request count" for dashboards → UDP. "Request count" for billing → TCP. Separate the pipelines.

Try It

Exercise: Warmup (10 minutes). For each: TCP or UDP? Online banking transactions. Spotify music streaming. A DNS lookup. A file download from S3. A multiplayer game sending player positions 60 times per second. Justify each choice.

Exercise: Core (25 minutes). Calculate the total overhead of establishing a new HTTPS connection cross-continent (TCP handshake + TLS 1.2 handshake). Then calculate for TLS 1.3. Then calculate for QUIC 0-RTT. Show why QUIC's UDP-based approach is faster for new connections.

Exercise: Stretch (40 minutes). Use Wireshark or tcpdump to capture a TCP connection (e.g., curl https://example.com). Identify: the three-way handshake (SYN, SYN-ACK, ACK), the data transfer (HTTP request/response), and the teardown (FIN, FIN-ACK). Count the round trips.

What's Next

Lesson 3.4 covers HTTP and HTTPS - the application protocol that carries virtually all web traffic, with its methods, status codes, headers, and TLS encryption layer.

Back to System Design Fundamentals