Free lesson · Computer Networking

Tcp Connection Termination

Easy10 min readnetworkingcomputer-networkstcp-ip

Closing a Connection Gracefully

Opening a TCP connection takes three messages. Closing one takes four. The extra step exists because TCP connections are full-duplex - data flows in both directions independently. Closing the connection in one direction does not close it in the other. Each side must independently signal that it has finished sending data, and each signal must be acknowledged.

Understanding TCP connection termination is not just academic. The TIME_WAIT state that follows a connection close is one of the most common operational issues in high-throughput server environments. Servers handling thousands of short-lived connections can accumulate tens of thousands of sockets in TIME_WAIT, consuming ephemeral ports and kernel memory. Knowing why TIME_WAIT exists and how to manage it is essential for anyone operating production systems.

The Four-Way Close (FIN → ACK → FIN → ACK)

TCP connection termination follows four messages:

Step 1 - FIN (Initiator → Responder)

The side that wants to close the connection sends a segment with the FIN (Finish) flag set. This says "I have no more data to send." The initiator enters the FIN-WAIT-1 state.

Importantly, FIN does not mean "I am done receiving." The initiator can still receive data from the other side. FIN is a one-directional shutdown.

Step 2 - ACK (Responder → Initiator)

The responder acknowledges the FIN with an ACK. The initiator moves to FIN-WAIT-2 and the responder moves to CLOSE-WAIT. At this point, the connection is half-closed: the initiator can no longer send data, but the responder can still send data to the initiator.

The responder remains in CLOSE-WAIT until its application finishes sending any remaining data and calls close() on the socket.

Step 3 - FIN (Responder → Initiator)

When the responder's application has finished sending data and closes the socket, the responder sends its own FIN. The responder moves to LAST-ACK.

Step 4 - ACK (Initiator → Responder)

The initiator acknowledges the responder's FIN. The responder moves to CLOSED and releases all resources. The initiator enters TIME_WAIT (discussed in detail below).

sequenceDiagram
    participant C as Client (Initiator)
    participant S as Server (Responder)

    Note over C,S: Connection ESTABLISHED

    C->>S: FIN<br/>"I'm done sending"
    Note over C: FIN-WAIT-1

    S->>C: ACK<br/>"Got your FIN"
    Note over C: FIN-WAIT-2
    Note over S: CLOSE-WAIT<br/>(can still send data)

    Note over S: App calls close()

    S->>C: FIN<br/>"I'm done sending too"
    Note over S: LAST-ACK

    C->>S: ACK<br/>"Got your FIN"
    Note over S: CLOSED
    Note over C: TIME-WAIT<br/>(wait 2×MSL)

    Note over C: After 2×MSL timeout
    Note over C: CLOSED

This sequence diagram shows the TCP four-way close. The client initiates by sending a FIN, moving to FIN-WAIT-1. The server acknowledges with an ACK, the client moves to FIN-WAIT-2, and the server enters CLOSE-WAIT. The server can still send data during CLOSE-WAIT. When the server's application calls close, the server sends its own FIN and moves to LAST-ACK. The client acknowledges, the server moves to CLOSED and releases all resources. The client enters TIME-WAIT, where it remains for twice the Maximum Segment Lifetime before finally moving to CLOSED.

The Half-Close State

The period between the first FIN-ACK exchange and the second FIN-ACK exchange is the half-close state. The connection is open in one direction and closed in the other. This is not a common pattern in most application protocols, but it has legitimate uses:

HTTP/1.0 pipelining: The client sends its request and then half-closes the connection (sends FIN), signaling to the server "I have no more data to send." The server reads the complete request, processes it, sends the response, and then closes its side.

Unix pipes over network: Programs like netcat use half-close to signal end-of-input while still receiving output.

In practice, most applications close both directions simultaneously - the application calls close() which sends a FIN, and the other side responds with both an ACK and its own FIN in quick succession (often combined into a single segment). The four-way close frequently appears as three segments in packet captures because the second ACK and second FIN are combined.

TIME_WAIT - The Most Important TCP State for Operations

After sending the final ACK, the connection initiator enters TIME_WAIT and remains there for 2 × MSL (Maximum Segment Lifetime). The MSL is the maximum time a segment can exist in the network (typically 60 seconds on Linux, making TIME_WAIT 120 seconds, though Linux actually uses 60 seconds total for TIME_WAIT).

Why TIME_WAIT Exists

TIME_WAIT serves two critical purposes:

Purpose 1 - Reliable connection termination. If the final ACK is lost, the responder will retransmit its FIN. The initiator must remain in TIME_WAIT to receive this retransmitted FIN and re-send the ACK. Without TIME_WAIT, the initiator would have moved to CLOSED and would respond to the retransmitted FIN with a RST, confusing the responder.

Purpose 2 - Preventing old segment confusion. After a connection closes, delayed segments from that connection may still be traveling through the network. If a new connection is immediately established using the same 4-tuple (source IP, source port, destination IP, destination port), these old segments could be mistakenly accepted as part of the new connection. TIME_WAIT ensures that all old segments expire (they have exceeded the MSL) before the 4-tuple can be reused.

TIME_WAIT in Production

TIME_WAIT becomes an operational problem on servers that handle many short-lived connections. The issue is most acute on servers that act as clients to downstream services - load balancers, API gateways, and microservice proxies - because the side that initiates the close enters TIME_WAIT, and these systems initiate closes frequently.

Port exhaustion. Each socket in TIME_WAIT occupies an ephemeral port. The ephemeral port range is typically 28,000-32,000 ports (configurable via net.ipv4.ip_local_port_range). A server creating and closing 500 connections per second to the same destination accumulates 500 × 60 = 30,000 sockets in TIME_WAIT - approaching or exceeding the ephemeral port range. New connections fail with "Cannot assign requested address" (EADDRNOTAVAIL) errors.

The exhaustion is actually per-destination: you can have up to ~28,000 TIME_WAIT sockets to each unique destination IP:port combination. A server connecting to many different backends is less likely to exhaust ports than one connecting repeatedly to a single backend. This is why connection pool misconfiguration to a single database server is a common trigger for port exhaustion.

Memory consumption. Each TIME_WAIT socket consumes kernel memory (approximately 150-300 bytes on Linux for a TIME_WAIT-specific mini-socket, significantly less than a full socket). At 100,000 TIME_WAIT sockets, this is roughly 15-30 MB - typically not a problem, but worth monitoring on systems with very high connection rates.

File descriptor consumption. TIME_WAIT sockets in Linux do not consume file descriptors - the socket has been closed from the application's perspective, and the kernel maintains only the minimal state needed for TIME_WAIT protection. This is a common misconception: TIME_WAIT does not reduce the number of file descriptors available to the application.

Managing TIME_WAIT

Several mechanisms help manage TIME_WAIT accumulation:

SO_REUSEADDR socket option. Allows a new socket to bind to a port that is in TIME_WAIT. This is essential for servers that restart - without it, the server cannot bind to its port for 60 seconds after shutdown because the old listening socket is in TIME_WAIT. Virtually every production server sets SO_REUSEADDR.

net.ipv4.tcp_tw_reuse (Linux). Allows outgoing connections to reuse TIME_WAIT sockets when the new connection's timestamp is greater than the old connection's last timestamp. This safely recycles ports for outgoing connections without the risks of the deprecated tcp_tw_recycle.

Connection pooling. The most effective mitigation by far. Instead of creating and closing a new TCP connection for each request, applications maintain a pool of persistent connections and reuse them. HTTP/2 multiplexes multiple requests over a single connection, dramatically reducing the number of connections opened and closed. A server using HTTP/1.1 without keep-alive creates one connection per request - each generating a TIME_WAIT. The same server using HTTP/2 might use a single long-lived connection for thousands of requests over minutes or hours, generating zero TIME_WAITs during normal operation.

Database connection pools (HikariCP for Java, pgbouncer for PostgreSQL) apply the same principle: a pool of 20 persistent connections serves thousands of queries without creating or destroying TCP connections. This is why connection pool sizing is one of the most important performance tuning decisions in application architecture.

Widen the ephemeral port range. sysctl -w net.ipv4.ip_local_port_range="1024 65535" expands the available ports from ~28,000 to ~64,000, roughly doubling capacity before exhaustion occurs.

Important deprecation note: The tcp_tw_recycle sysctl option was historically used to aggressively recycle TIME_WAIT sockets but was removed from Linux 4.12 (2017) because it caused silent connection failures when clients were behind NAT. Since NAT causes multiple clients to share an IP address, timestamp-based recycling could reject legitimate connections. Never use tcp_tw_recycle - use tcp_tw_reuse instead, which is safe for outgoing connections.

CLOSE-WAIT Leaks - The Application Bug Signal

While TIME_WAIT is normal and expected, CLOSE-WAIT accumulation is almost always an application bug. A socket in CLOSE-WAIT means the remote side has sent a FIN (it is done sending), but the local application has not called close() on its socket.

Common causes of CLOSE-WAIT leaks:

Resource leak in application code. The application opens connections but fails to close them in error paths. A common pattern in languages without automatic resource cleanup: the application reads a response, encounters an exception during processing, and the error handler does not close the socket. The socket remains in CLOSE-WAIT indefinitely, consuming file descriptors and memory.

Blocked application threads. The application is processing data and has not yet reached the close() call. If the application is blocked waiting for a database query, disk I/O, or a downstream service, sockets accumulate in CLOSE-WAIT until the blocking resolves.

Missing connection cleanup in connection pools. A connection pool that does not properly detect when the remote side closes a connection may hold dead connections in CLOSE-WAIT.

Diagnosing CLOSE-WAIT leaks: ss -tnp state close-wait (Linux) shows all CLOSE-WAIT sockets with the process that owns them. If CLOSE-WAIT sockets are growing over time for a specific process, that process has a resource leak that needs fixing.

Graceful Shutdown in Production

Properly shutting down a server is more complex than simply closing all sockets. A graceful shutdown must:

Stop accepting new connections - close the listening socket or remove the server from the load balancer
Complete in-flight requests - allow currently active requests to finish processing and send their responses
Close idle connections - send FIN on persistent connections that have no active requests
Wait with a deadline - give in-flight requests a reasonable timeout (e.g., 30 seconds) and then forcefully close remaining connections with RST

Kubernetes implements this pattern through its pod lifecycle: when a pod is terminated, it receives a SIGTERM signal, is removed from the service's endpoints (stopping new traffic), and has a configurable terminationGracePeriodSeconds (default 30 seconds) to complete in-flight work before receiving SIGKILL. Applications that do not handle SIGTERM for graceful shutdown risk dropping active requests during deployments.

RST - The Abortive Close

Not all connections end gracefully. A RST (Reset) segment immediately terminates a connection without the four-way close process. RST is used when:

An application crashes without closing its sockets
A connection attempt reaches a port with no listening application
A firewall decides to terminate a connection
An application explicitly aborts with SO_LINGER set to zero

RST is not an error in the protocol - it is a valid mechanism for immediate connection termination. But it is ungraceful: any data in the sender's or receiver's buffers is lost, the other side receives an error ("Connection reset by peer"), and there is no TIME_WAIT protection against old segment confusion. Applications should use graceful FIN-based closing for normal operations and reserve RST for exceptional situations.

In packet captures, RST segments are common and not always concerning. A client connecting to a closed port receives RST (this is normal - it means no application is listening). A firewall timing out an idle connection sends RST to both sides (this is expected for long-idle connections). A load balancer health check that connects and immediately disconnects may use RST to avoid TIME_WAIT accumulation. Understanding when RST is normal versus when it indicates a problem is an important debugging skill.

graph TD
    subgraph States["TCP Connection Termination States"]
        direction TB
        EST["ESTABLISHED"] -->|"Send FIN"| FW1["FIN-WAIT-1"]
        FW1 -->|"Receive ACK"| FW2["FIN-WAIT-2"]
        FW2 -->|"Receive FIN"| TW["TIME-WAIT<br/>(2 × MSL)"]
        TW -->|"Timeout"| CLOSED1["CLOSED"]

        EST -->|"Receive FIN"| CW["CLOSE-WAIT"]
        CW -->|"Send FIN"| LA["LAST-ACK"]
        LA -->|"Receive ACK"| CLOSED2["CLOSED"]
    end

    style EST fill:#4f46e5,stroke:#4338ca,color:#fff
    style FW1 fill:#7c3aed,stroke:#6d28d9,color:#fff
    style FW2 fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style TW fill:#ef4444,stroke:#dc2626,color:#fff
    style CLOSED1 fill:#22c55e,stroke:#16a34a,color:#fff
    style CW fill:#a78bfa,stroke:#8b5cf6,color:#fff
    style LA fill:#c4b5fd,stroke:#a78bfa,color:#000
    style CLOSED2 fill:#22c55e,stroke:#16a34a,color:#fff

This diagram shows the TCP termination state machine for both sides. The initiator transitions from ESTABLISHED to FIN-WAIT-1 by sending FIN, then to FIN-WAIT-2 on receiving ACK, then to TIME-WAIT on receiving the responder's FIN, and finally to CLOSED after the 2 times MSL timeout. The responder transitions from ESTABLISHED to CLOSE-WAIT on receiving FIN, then to LAST-ACK on sending its own FIN, and finally to CLOSED on receiving the final ACK. TIME-WAIT, shown in red, is the operationally significant state where the initiator holds the socket open to prevent old segment confusion.

Interview: TIME_WAIT is one of the most practically relevant TCP topics:

"Your service is throwing 'Cannot assign requested address' errors under load." - Likely ephemeral port exhaustion from TIME_WAIT accumulation. Check with ss -s or netstat -an | grep TIME_WAIT | wc -l. Mitigations: enable tcp_tw_reuse, widen the port range, and most importantly, use connection pooling to reduce connection churn.

"Why does my server take 60 seconds to restart?" - The old socket is in TIME_WAIT. The new server process cannot bind to the same port. Solution: set SO_REUSEADDR on the listening socket before binding.

"Why is there a TIME_WAIT state? Can we just skip it?" - TIME_WAIT ensures the final ACK is reliably delivered and prevents old segments from interfering with new connections on the same 4-tuple. Skipping it (via RST or the deprecated tcp_tw_recycle) risks data corruption on new connections. The proper solution is to reduce the number of connections being created and destroyed, not to eliminate TIME_WAIT.

What Comes Next

The next topic covers TCP head-of-line blocking - the fundamental ordering constraint that limits TCP's performance for multiplexed protocols like HTTP/2, and why QUIC was built on UDP to eliminate it.

Exercise: Starter: Count TIME_WAIT sockets on your machine. On Linux: ss -s shows summary statistics including TIME_WAIT count. On Mac: netstat -an | grep TIME_WAIT | wc -l. If you have few TIME_WAIT sockets, generate some by running curl repeatedly: for i in $(seq 100); do curl -s https://google.com > /dev/null; done and check again immediately.

Intermediate: Capture a TCP connection close with tcpdump or Wireshark. Filter by tcp.flags.fin == 1 to find FIN segments. Can you see the four-way close? Or does it appear as three segments (ACK+FIN combined)? Which side initiated the close? What is the TIME_WAIT duration on your system? Check with sysctl net.ipv4.tcp_fin_timeout (Linux).

Advanced: Research the SO_LINGER socket option. How does linger.l_onoff = 1, l_linger = 0 cause an RST instead of a graceful FIN close? When would you intentionally use this? How does Nginx's reset_timedout_connection directive use RST to avoid TIME_WAIT for timed-out connections? What are the risks of this approach?

Back to Computer Networking