Free lesson · Computer Networking

Tcp Flow Control Sliding Window

Easy10 min readnetworkingcomputer-networkstcp-ip

Matching the Sender to the Receiver

TCP reliability guarantees that every byte reaches the receiver. But what if the sender is faster than the receiver? A high-performance server sending data to a mobile device on a slow connection could flood the receiver's buffer, causing the receiver to drop packets and triggering a cascade of retransmissions. The network would be wasting bandwidth sending data that the receiver cannot process.

Flow control solves this problem. It is a mechanism that allows the receiver to control how fast the sender transmits, ensuring the sender never overwhelms the receiver's buffer. Flow control is a receiver-side protection - it is about the receiver's capacity, not the network's capacity (that is congestion control, covered in the next topic).

TCP implements flow control using a sliding window mechanism. The receiver advertises how much buffer space it has available, and the sender limits its transmissions to fit within that window. As the receiver processes data and frees buffer space, the window slides forward, allowing the sender to transmit more.

The Receive Window

Every TCP segment includes a Window Size field (16 bits, or more with window scaling) that tells the sender how many bytes the receiver can accept beyond the last acknowledged byte. This is the receive window (rwnd).

If the receiver has 64 KB of buffer space and has received 10 KB that the application has not yet read, the receive window is 54 KB - the remaining free space. The receiver advertises rwnd = 54 KB in every ACK it sends. The sender must not have more than 54 KB of unacknowledged data in flight at any time.

As the application reads data from the receive buffer, space frees up, and the receiver advertises a larger window in subsequent ACKs. As the sender transmits more data without receiving ACKs, the window shrinks. This dynamic adjustment is what makes it a "sliding" window - the window slides along the byte stream as data is sent, acknowledged, and consumed.

Zero Window

If the receiver's buffer fills up completely (the application is not reading data fast enough), the receiver advertises rwnd = 0. The sender must stop transmitting data entirely. This is called a zero window condition.

The sender does not wait indefinitely - it periodically sends window probe segments (1-byte segments) to check if the window has reopened. When the receiver's application reads data and frees buffer space, the receiver responds to the probe with an updated non-zero window, and the sender resumes transmission.

Zero window conditions occur in practice when the receiving application is under heavy CPU load, blocked on I/O, or performing garbage collection. They are visible in packet captures and are a useful diagnostic signal - a zero window from the receiver indicates the bottleneck is the receiving application, not the network.

sequenceDiagram
    participant S as Sender
    participant R as Receiver<br/>(Buffer: 4000 bytes)

    Note over R: Buffer empty<br/>rwnd = 4000

    S->>R: Seq=0, 1000 bytes
    Note over R: Buffer: 1000/4000 used<br/>rwnd = 3000
    R->>S: ACK=1000, rwnd=3000

    S->>R: Seq=1000, 1000 bytes
    S->>R: Seq=2000, 1000 bytes
    S->>R: Seq=3000, 1000 bytes
    Note over R: Buffer: 4000/4000 FULL<br/>rwnd = 0
    R->>S: ACK=4000, rwnd=0

    Note over S: STOP! Window is zero<br/>Cannot send more data

    Note over R: App reads 2000 bytes<br/>rwnd = 2000
    S->>R: Window Probe (1 byte)
    R->>S: ACK=4000, rwnd=2000
    Note over S: Window reopened!<br/>Resume sending

This sequence diagram shows TCP flow control with a sliding window. The receiver has a 4000-byte buffer. The sender transmits segments, and after each the receiver advertises a smaller window as the buffer fills up. After four 1000-byte segments, the buffer is full and the receiver advertises window size zero. The sender stops transmitting. When the receiving application reads 2000 bytes from the buffer, the receiver responds to a window probe with an updated window of 2000, and the sender resumes transmission.

The Sliding Window in Detail

The sender maintains a view of the byte stream divided into four regions:

The "window" is the combination of regions 2 and 3 - the range of bytes the sender is allowed to have outstanding. As ACKs arrive, region 1 grows (more bytes are confirmed), region 2 shrinks (fewer bytes are in flight), and the window slides forward, making new bytes available in region 3.

The Effective Window

In practice, the sender's transmission rate is limited by the minimum of two windows:

Receive window (rwnd): How much data the receiver can accept. Set by the receiver via the Window Size field.

Congestion window (cwnd): How much data the network can handle. Set by the sender's congestion control algorithm (covered in the next topic).

The sender can have at most min(rwnd, cwnd) bytes in flight at any time. Flow control (rwnd) prevents overwhelming the receiver. Congestion control (cwnd) prevents overwhelming the network. Both must be satisfied simultaneously.

Silly Window Syndrome

A pathological interaction can occur when the receiver frees buffer space in very small increments. The receiver advertises a tiny window (say, 10 bytes), the sender transmits a tiny segment (10 bytes of data with 40+ bytes of headers), the receiver processes it and advertises another tiny window, and the cycle continues. This is called silly window syndrome (SWS) - the connection becomes dominated by header overhead, with extremely poor throughput.

SWS is prevented through two complementary mechanisms:

Receiver-side prevention (Clark's algorithm): The receiver does not advertise a non-zero window until the available space is at least MSS or half the buffer size, whichever is smaller. Instead, it keeps advertising rwnd = 0 until enough space accumulates to be worth advertising. This prevents the receiver from tempting the sender into transmitting tiny segments.

Sender-side prevention (Nagle's algorithm): As discussed in the previous topic, the sender buffers small data until an ACK arrives or MSS bytes accumulate. This prevents the sender from transmitting tiny segments even if the receiver advertises a small window.

Together, these mechanisms ensure that TCP segments carry meaningful amounts of data relative to their header overhead, maintaining efficient bandwidth utilization.

Delayed ACKs and Flow Control

TCP receivers do not always send an ACK immediately for every received segment. Delayed ACKs (RFC 1122) allow the receiver to wait up to 200ms (typically 40ms in Linux) before sending an ACK, in the hope that the ACK can be piggybacked on a data segment the receiver is about to send. This reduces the number of pure ACK segments on the network.

However, delayed ACKs interact with flow control in subtle ways. If the sender is waiting for an ACK to slide the window forward, and the receiver is delaying the ACK, both sides are waiting for each other - creating a brief stall. The maximum delay is bounded (200ms by spec, typically shorter in practice), so the stall resolves quickly, but it can be noticeable in latency-sensitive applications.

The interaction between delayed ACKs and Nagle's algorithm is particularly problematic: the sender buffers data waiting for an ACK (Nagle's), and the receiver delays the ACK waiting for more data or a timeout (delayed ACKs). The combined delay can reach 200ms - unacceptable for interactive applications. This is another reason modern servers set TCP_NODELAY (disabling Nagle's) and sometimes TCP_QUICKACK (disabling delayed ACKs) on latency-sensitive connections.

Flow Control in High-Performance Scenarios

Flow control becomes particularly important in several production scenarios:

Database replication over WAN links. A primary database sending write-ahead logs to a replica across a high-latency link needs large TCP buffers to maintain throughput. If the receive buffer is too small, the window closes before enough data is in flight to fill the pipe, and replication lag increases. This is a common tuning issue in cross-region PostgreSQL and MySQL replication setups.

Large file transfers. When transferring multi-gigabyte files, the TCP window must be large enough to keep the connection fully utilized. Tools like iperf3 measure achievable TCP throughput and help identify when buffer sizes are limiting performance.

Streaming data pipelines. Kafka consumers reading from remote brokers, gRPC streams, and WebSocket connections for real-time data all depend on properly sized TCP windows. Undersized buffers cause the window to close frequently, creating stop-and-go behavior that reduces throughput and increases latency variability.

Container memory limits. In Kubernetes environments, pods have memory limits enforced by cgroups. TCP buffer memory counts toward the pod's memory usage. If the pod's memory limit is tight, the kernel may not be able to allocate sufficient TCP buffers, causing small receive windows and poor throughput. This is a subtle failure mode where the application appears to have a "network problem" but the root cause is a memory constraint limiting TCP buffer allocation.

Window Scaling for Modern Networks

The TCP header's Window Size field is 16 bits, allowing a maximum advertised window of 65,535 bytes (64 KB). For a connection with 100ms RTT, this limits throughput to approximately:

Throughput = Window Size / RTT = 65,535 / 0.1 = 655,350 bytes/sec ≈ 5.2 Mbps

This was adequate in 1981 when TCP was designed, but is absurdly low for modern networks capable of 10-100 Gbps. A 10 Gbps connection with 100ms RTT needs a window of approximately 125 MB to fully utilize the bandwidth.

Window scaling (RFC 7323), negotiated during the three-way handshake, extends the effective window size. The scale factor (0-14) is applied as a left bit shift to the Window Size field. A scale factor of 7 means the actual window size is the advertised value multiplied by 128 (2^7), allowing windows up to 65,535 × 16,384 = 1,073,725,440 bytes (~1 GB) with scale factor 14.

Modern operating systems negotiate window scaling automatically. Linux defaults to a scale factor calculated based on the configured maximum receive buffer size (net.core.rmem_max). If you examine a packet capture, you will see the window scale option in the SYN and SYN-ACK segments.

Bandwidth-Delay Product

The bandwidth-delay product (BDP) is the maximum amount of data that can be "in flight" on a network path at any given time:

BDP = Bandwidth × RTT

For a 1 Gbps connection with 50ms RTT:

BDP = 1,000,000,000 bits/sec × 0.050 sec = 50,000,000 bits = 6.25 MB

To fully utilize this connection, the TCP window must be at least 6.25 MB. If the window is smaller than the BDP, the sender is forced to stop and wait for ACKs, and the connection is bandwidth-underutilized. Understanding BDP is essential for tuning TCP performance on high-bandwidth, high-latency links - exactly the type of connection used for cross-region cloud communication.

Some real-world BDP examples:

Notice how the required window grows dramatically with RTT. Within a data center, even a small window provides enough capacity. Across continents, multi-megabyte windows are necessary to fully utilize the link. If your cross-region replication is slow despite having a fast link, the first thing to check is whether the TCP window is large enough to fill the pipe.

Long Fat Networks (LFNs)

Networks with large BDP are called Long Fat Networks (LFNs, pronounced "elephants"). They pose unique challenges for TCP because the standard 64 KB window is far too small, sequence numbers can wrap around during a single connection (requiring PAWS), and lost packets have a disproportionate impact because retransmission takes a full RTT. High-speed intercontinental links, satellite connections, and dedicated WAN circuits are all LFNs that require careful TCP tuning.

graph TD
    subgraph WindowTypes["Two Windows Limiting the Sender"]
        direction TB
        RWND["Receive Window (rwnd)<br/>Set by RECEIVER<br/>Prevents overwhelming<br/>the receiver's buffer"]
        CWND["Congestion Window (cwnd)<br/>Set by SENDER<br/>Prevents overwhelming<br/>the network"]
        EFF["Effective Window<br/>= min(rwnd, cwnd)<br/>Actual transmission limit"]
        RWND --> EFF
        CWND --> EFF
    end

    style RWND fill:#4f46e5,stroke:#4338ca,color:#fff
    style CWND fill:#7c3aed,stroke:#6d28d9,color:#fff
    style EFF fill:#22c55e,stroke:#16a34a,color:#fff

This diagram shows the two windows that limit a TCP sender. The receive window, set by the receiver, prevents overwhelming the receiver's buffer. The congestion window, set by the sender's congestion control algorithm, prevents overwhelming the network. The effective window - the actual transmission limit - is the minimum of the two. Both constraints must be satisfied simultaneously.

Interview: Flow control concepts appear in performance and System Design discussions:

"Why is throughput low on a high-latency connection?" - The TCP window may be smaller than the bandwidth-delay product (BDP). If the window cannot fill the pipe, the sender must pause and wait for ACKs, underutilizing the available bandwidth. Increasing the receive buffer (and thus the advertised window) can improve throughput.

"What causes a zero window?" - The receiving application is not reading data from the TCP buffer fast enough. The buffer fills, the receiver advertises rwnd=0, and the sender stops. This is an application-level bottleneck, not a network problem. Investigate why the receiver is slow - CPU load, slow disk I/O, garbage collection pauses, or blocked threads.

"How do you tune TCP for a cross-region database replication link?" - Increase the send and receive buffer sizes to accommodate the BDP. For a 1 Gbps link with 100ms RTT, you need at least 12.5 MB of buffer. On Linux: net.core.rmem_max, net.core.wmem_max, and net.ipv4.tcp_rmem/tcp_wmem control these values.

What Comes Next

The next topic covers TCP congestion control - the mechanism that prevents TCP from overwhelming the network itself. While flow control matches the sender to the receiver's capacity, congestion control matches the sender to the network's capacity. Together, they determine TCP's actual throughput.

Exercise: Starter: Check your system's TCP buffer sizes. On Linux: sysctl net.ipv4.tcp_rmem and sysctl net.ipv4.tcp_wmem. These show the minimum, default, and maximum buffer sizes. Calculate the maximum achievable throughput on a 50ms RTT connection given your maximum receive buffer.

Intermediate: Use ss -i on Linux to examine active TCP connections. Find the rcv_space (receive buffer) and snd_wnd (send window) values for an active connection. Can you identify the window scaling factor? How does the effective window compare to the bandwidth-delay product of the connection?

Advanced: Research TCP autotuning in Linux. Modern kernels automatically adjust the receive buffer size based on the connection's BDP, up to the configured maximum (net.ipv4.tcp_rmem third value). How does the kernel estimate BDP? What happens when autotuning is disabled? Why might a container orchestrator or cgroup limit affect TCP performance by restricting memory available for socket buffers?

Back to Computer Networking