Free lesson · Computer Networking
Tcp Reliable Data Transfer
The Promise TCP Makes
TCP promises that every byte sent by the sender will be received by the receiver, in the correct order, without corruption. This is a remarkable guarantee - the underlying IP network provides none of these things. IP packets can be lost, duplicated, reordered, or corrupted in transit. TCP builds reliability on top of an unreliable network using four mechanisms: acknowledgments, retransmission, duplicate detection, and checksums.
Understanding how TCP achieves reliability is essential for two reasons. First, it explains why TCP has overhead compared to UDP - reliability is not free. Second, it reveals the failure modes that affect real systems: retransmission delays that spike latency, timeout calculations that affect responsiveness, and the reality that TCP provides "at-least-once" delivery rather than the mythical "exactly-once" delivery that many engineers assume.
Sequence Numbers and Acknowledgments
TCP treats data as a continuous byte stream, not a sequence of independent messages. Each byte in the stream is assigned a number, starting from the Initial Sequence Number (ISN) established during the handshake. When TCP sends a segment, the Sequence Number field in the header indicates the byte number of the first byte in that segment's payload.
The receiver acknowledges data using the Acknowledgment Number field, which indicates the next byte the receiver expects. If the receiver has received bytes 0 through 999, it sends an ACK with acknowledgment number 1000 - meaning "I have received everything up to byte 999 and am ready for byte 1000."
This is a cumulative acknowledgment system. ACK number 1000 means all bytes before 1000 have been received - it does not just acknowledge a single segment. This simplifies the protocol but creates a problem: if segments 1, 2, and 4 arrive but segment 3 is lost, the receiver can only acknowledge up to segment 2. It cannot tell the sender that segment 4 arrived - the sender must retransmit segment 3 and possibly segments 4+ as well.
Selective Acknowledgment (SACK) addresses this limitation. With SACK enabled (negotiated during the handshake), the receiver can specify exactly which byte ranges it has received, even if they are non-contiguous. The sender can then retransmit only the missing segments, avoiding unnecessary retransmissions. SACK is enabled by default on virtually all modern operating systems.
sequenceDiagram
participant S as Sender
participant R as Receiver
S->>R: Seq=0, Data: bytes 0-499<br/>(500 bytes)
S->>R: Seq=500, Data: bytes 500-999<br/>(500 bytes)
R->>S: ACK=1000<br/>"I have everything up to byte 999"
S->>R: Seq=1000, Data: bytes 1000-1499
Note over R: Segment 1000-1499 received
S->>R: Seq=1500, Data: bytes 1500-1999
Note over S,R: Segment lost in transit!
S->>R: Seq=2000, Data: bytes 2000-2499
Note over R: Gap detected!<br/>Has 1000-1499 and 2000-2499<br/>Missing 1500-1999
R->>S: ACK=1500<br/>SACK: 2000-2499<br/>"I need byte 1500, but<br/>I already have 2000-2499"
S->>R: Seq=1500, Data: bytes 1500-1999<br/>(retransmit only the gap)
R->>S: ACK=2500<br/>"Now I have everything"
This sequence diagram shows TCP reliable data transfer with SACK. The sender transmits segments covering bytes 0 through 2499. The segment covering bytes 1500 to 1999 is lost in transit. The receiver detects the gap - it has bytes up to 1499 and bytes 2000 to 2499, but is missing 1500 to 1999. The receiver sends a cumulative ACK of 1500 (the next byte it needs) along with a SACK block indicating it already has bytes 2000 to 2499. The sender retransmits only the missing segment from 1500 to 1999. The receiver then acknowledges everything up to byte 2500.
Retransmission Timers and RTT Estimation
When the sender transmits a segment, it starts a retransmission timer. If an acknowledgment is not received before the timer expires, the sender assumes the segment was lost and retransmits it. The critical question is: how long should the timer be?
Too short: the sender retransmits segments that are still in transit, wasting bandwidth and potentially causing congestion. Too long: the sender waits unnecessarily before recovering from actual losses, increasing latency.
TCP solves this by continuously estimating the round-trip time (RTT) and setting the retransmission timeout (RTO) based on the estimate. The algorithm (RFC 6298) maintains two variables:
SRTT (Smoothed RTT): A weighted moving average of measured RTTs. Each new measurement is blended with the previous estimate:
SRTT = (1 - α) × SRTT + α × measured_RTT (where α = 1/8)
RTTVAR (RTT Variance): A measure of how much the RTT fluctuates:
RTTVAR = (1 - β) × RTTVAR + β × |SRTT - measured_RTT| (where β = 1/4)
The retransmission timeout is set to:
RTO = SRTT + 4 × RTTVAR
This formula adapts to network conditions. On a stable LAN with 1ms RTT and low variance, the RTO might be 5ms. On a congested intercontinental link with 200ms RTT and high variance, the RTO might be 800ms. The RTO also has a minimum value (typically 200ms in Linux, though some implementations use 1 second) and a maximum value (typically 120 seconds).
Exponential backoff: When a retransmission timeout fires, TCP doubles the RTO for subsequent retransmissions of the same segment. This prevents the sender from aggressively retransmitting into a congested network. If the first RTO is 400ms, the second attempt uses 800ms, the third uses 1600ms, and so on. The doubling continues until the maximum RTO is reached or the connection gives up (typically after 5-15 retransmissions, depending on OS settings).
Fast Retransmit
Waiting for the retransmission timer to expire before retransmitting a lost segment can introduce significant delay - potentially hundreds of milliseconds. Fast retransmit is an optimization that detects packet loss more quickly.
When the receiver detects a gap in the byte stream (it receives a segment with a sequence number higher than expected), it immediately sends a duplicate ACK - an ACK with the same acknowledgment number as the last in-order byte received. If the sender receives three consecutive duplicate ACKs for the same byte, it assumes the next segment was lost and retransmits it immediately, without waiting for the retransmission timer to expire.
Why three duplicate ACKs, not one? Because a single duplicate ACK might result from packet reordering in the network (a common occurrence), not loss. Three duplicate ACKs provide stronger evidence that the segment is actually lost rather than simply arriving out of order.
Fast retransmit dramatically reduces recovery time. Instead of waiting for an RTO timeout (potentially hundreds of milliseconds), the sender detects loss after receiving three duplicate ACKs (typically within a few milliseconds of the loss occurring) and retransmits immediately.
Duplicate Detection
Because TCP retransmits segments, the receiver may receive the same data twice - once from the original transmission and once from the retransmission. TCP must detect and discard duplicates to maintain its exactly-once delivery guarantee to the application.
Duplicate detection is straightforward: each byte in the stream has a unique sequence number. If the receiver receives data with a sequence number it has already received and acknowledged, it discards the duplicate and re-sends the acknowledgment. The sequence number space is large enough (32 bits, wrapping around at 2^32) that duplicates from old connections are extremely unlikely - and the PAWS mechanism (Protection Against Wrapped Sequences, using TCP timestamps) handles the edge case of sequence number wraparound on very high-speed connections.
Checksums - Detecting Corruption
TCP includes a 16-bit checksum that covers the TCP header, the data payload, and a pseudo-header derived from the IP header (source IP, destination IP, protocol number, TCP length). The sender computes the checksum before transmitting, and the receiver recomputes it upon arrival. If the checksums do not match, the segment is silently discarded - the sender eventually detects the loss through the retransmission timeout or duplicate ACKs and resends the data.
The TCP checksum catches most random bit errors caused by noisy links, memory corruption in network equipment, or software bugs. However, it is only a 16-bit checksum (not a cryptographic hash), so it cannot detect all errors - research has shown that approximately 1 in 10 billion packets with bit errors passes the TCP checksum undetected. For applications that require stronger integrity guarantees (database replication, financial transactions), application-level checksums or TLS (which adds its own integrity verification) provide additional protection.
Nagle's Algorithm - Reducing Small Packet Overhead
Without optimization, an interactive application like SSH would send individual keystrokes as tiny TCP segments - 1 byte of data with 40+ bytes of TCP/IP headers. This is extremely wasteful and can congest the network with many tiny packets.
Nagle's algorithm (RFC 896) addresses this by buffering small segments. The rule is simple: if there is unacknowledged data in flight, buffer new small data instead of sending it immediately. Send the buffered data either when an ACK arrives (confirming the network can handle more data) or when the buffer reaches the MSS.
Nagle's algorithm works well for bulk data transfers (it has no effect because segments are already MSS-sized) and reduces overhead for interactive applications. However, it can cause latency problems for protocols that send small messages and expect immediate responses - particularly when combined with delayed ACKs (where the receiver waits up to 200ms before sending an ACK, hoping to piggyback it on a response). The combination of Nagle's algorithm and delayed ACKs can create a 200ms delay on small messages, which is unacceptable for real-time protocols.
Applications that need low latency for small messages disable Nagle's algorithm using the TCP_NODELAY socket option. Most modern web servers and database clients set TCP_NODELAY by default.
Retransmission in Practice
In production systems, TCP retransmission behavior affects application performance in measurable ways:
Tail latency. When a TCP segment is lost on a request-response communication (like an HTTP API call), the response is delayed by at least one RTO period (typically 200ms minimum). This means that even with a 0.1% packet loss rate, approximately 1 in 1,000 requests will experience at least 200ms of additional latency due to retransmission. This is a major contributor to tail latency (P99, P99.9) in microservice architectures where a single user request may traverse dozens of services - the probability that at least one hop experiences a retransmission grows multiplicatively.
Head-of-line blocking from retransmission. When a segment is lost, all subsequent data - even data that has already been received and buffered by the receiver - cannot be delivered to the application until the lost segment is retransmitted and received. This is because TCP guarantees ordered delivery. The application sees a latency spike equal to the retransmission delay for all data in the affected stream, not just the lost segment. This is one of TCP's fundamental limitations, discussed further in Topic 7.
Connection reset after repeated failures. If TCP exhausts its retransmission attempts (controlled by tcp_retries2 on Linux, default 15 attempts over approximately 13-30 minutes depending on RTO), it gives up and resets the connection. The application receives a connection error. In production, this typically means the destination is truly unreachable - but the long timeout before giving up can leave connections in a "zombie" state for minutes, consuming resources on the sender.
The "Exactly-Once" Myth
TCP is often described as providing "exactly-once delivery." This is misleading. TCP guarantees that the application receives each byte of data at least once, in order, without corruption. But at the TCP level, the same segment may be transmitted multiple times (due to retransmission) and acknowledged multiple times (due to delayed or duplicate ACKs).
The more accurate description is at-least-once delivery with deduplication. TCP sends data at least once and deduplicates at the receiver, achieving the effect of exactly-once from the application's perspective. But the distinction matters when designing systems:
Idempotency. Because TCP may retransmit requests, application-level protocols built on TCP should ideally be idempotent - processing the same request twice should produce the same result. HTTP GET is idempotent by design. HTTP POST is not inherently idempotent, which is why APIs use idempotency keys for operations like payment processing.
TCP connection failure. If a TCP connection is interrupted after the sender transmits data but before the sender receives an acknowledgment, the sender does not know if the data was received. It may have been received and processed by the application, or it may have been lost. This ambiguity is fundamental - no protocol can eliminate it entirely. It is the basis of the Two Generals' Problem in distributed systems.
graph TD
subgraph Reliability["TCP Reliability Mechanisms"]
direction TB
SEQ["Sequence Numbers<br/>Track byte position<br/>in the stream"]
ACK["Acknowledgments<br/>Confirm receipt of<br/>data bytes"]
RTO["Retransmission Timer<br/>Resend if ACK not<br/>received in time"]
FAST["Fast Retransmit<br/>3 duplicate ACKs<br/>= immediate resend"]
SACK["Selective ACK<br/>Report non-contiguous<br/>received ranges"]
CSUM["Checksum<br/>Detect corrupted<br/>data in transit"]
end
style SEQ fill:#4f46e5,stroke:#4338ca,color:#fff
style ACK fill:#6366f1,stroke:#4f46e5,color:#fff
style RTO fill:#7c3aed,stroke:#6d28d9,color:#fff
style FAST fill:#8b5cf6,stroke:#7c3aed,color:#fff
style SACK fill:#a78bfa,stroke:#8b5cf6,color:#fff
style CSUM fill:#c4b5fd,stroke:#a78bfa,color:#000
This diagram shows the six reliability mechanisms TCP uses. Sequence numbers track byte positions in the stream. Acknowledgments confirm receipt of data. The retransmission timer triggers resending if an ACK is not received within the timeout period. Fast retransmit uses three duplicate ACKs as a signal to resend immediately without waiting for the timer. Selective ACK reports non-contiguous received ranges so the sender can retransmit only missing data. And checksums detect data corruption in transit.
Interview: TCP reliability is frequently tested in System Design and networking interviews:
"How does TCP guarantee delivery?" - Through acknowledgments and retransmission. The sender keeps data in a retransmission buffer until acknowledged. If an ACK is not received within the RTO, the data is retransmitted. SACK allows selective retransmission of only the missing segments.
"What happens if a TCP segment is lost?" - Two detection mechanisms: the retransmission timer expires (slow path), or the sender receives three duplicate ACKs triggering fast retransmit (fast path). Either way, the lost segment is retransmitted.
"Does TCP provide exactly-once delivery?" - Effectively yes, from the application's perspective - TCP deduplicates at the receiver. But at the protocol level, it is at-least-once with deduplication. And at the application level, if the connection fails mid-transfer, the sender cannot know whether the last segment was received. This is why application-level idempotency matters.
What Comes Next
The next topic covers TCP flow control - the sliding window mechanism that prevents a fast sender from overwhelming a slow receiver. Flow control is about matching the sender's rate to the receiver's capacity, which is a different problem from congestion control (matching the sender's rate to the network's capacity, covered in Topic 5).
Exercise: Starter: Start a file download and monitor TCP retransmissions. On Linux:
netstat -s | grep retransmitshows the system-wide retransmission count. Run it before and after a large download and calculate how many segments were retransmitted. What percentage of total segments were retransmissions?Intermediate: Use
tcpdumpor Wireshark to capture a TCP connection and identify retransmissions. In Wireshark, the filtertcp.analysis.retransmissionhighlights retransmitted segments. Can you see the duplicate ACKs that triggered fast retransmit? What was the RTT and how does it compare to the observed retransmission delay?Advanced: Research the relationship between TCP retransmission and application-level timeouts. If your HTTP client has a 5-second timeout and TCP's RTO is 400ms with exponential backoff, how many retransmission attempts will TCP make before the application timeout fires? What happens if the application cancels the request while TCP is still retransmitting - does TCP send a RST? How does this interact with connection pooling in HTTP/2?