AlgoInsight

Free lesson · Computer Networking

Nat In Routing Context

Easy10 min readnetworkingcomputer-networkstcp-ip

NAT as a Routing Function

Module 03 introduced NAT as an address conservation mechanism - multiple devices sharing a single public IP. This topic examines NAT from a different angle: how NAT interacts with the routing process itself. When a router performs NAT, it is not just translating addresses - it is making routing decisions, maintaining connection state, and interacting with firewall rules in ways that profoundly affect network architecture, particularly in cloud and microservice environments.

Understanding NAT in the routing context matters because modern production infrastructure uses NAT extensively. Every AWS NAT Gateway, every Kubernetes cluster with IP masquerading, every Docker container accessing the internet, and every corporate network with private addressing involves NAT as part of the routing path. When traffic takes an unexpected path, when latency spikes occur at NAT boundaries, or when connections fail asymmetrically, understanding how NAT operates within the routing pipeline is essential for diagnosis.

Where NAT Fits in the Routing Pipeline

NAT occurs at a specific point in the packet processing pipeline on a router or firewall. The exact placement matters because it determines whether the forwarding table lookup uses the original address or the translated address.

Pre-Routing NAT (DNAT - Destination NAT)

Destination NAT translates the destination address before the routing decision. When an incoming packet arrives at the NAT device, the destination IP is changed first, and then the forwarding table lookup uses the new destination to determine where to send the packet.

Common uses of DNAT:

Post-Routing NAT (SNAT - Source NAT)

Source NAT translates the source address after the routing decision. The forwarding table lookup uses the original destination to determine the outgoing interface, and then the source address is changed before the packet leaves.

Common uses of SNAT:

graph TD
    subgraph Pipeline["NAT in the Routing Pipeline"]
        direction TB
        PKT["Incoming Packet"] --> DNAT["Pre-routing DNAT<br/>(Translate destination<br/>BEFORE routing)"]
        DNAT --> ROUTE["Routing Decision<br/>(Forwarding table<br/>lookup)"]
        ROUTE --> SNAT["Post-routing SNAT<br/>(Translate source<br/>AFTER routing)"]
        SNAT --> OUT["Outgoing Packet"]
    end

    style PKT fill:#a78bfa,stroke:#8b5cf6,color:#fff
    style DNAT fill:#4f46e5,stroke:#4338ca,color:#fff
    style ROUTE fill:#7c3aed,stroke:#6d28d9,color:#fff
    style SNAT fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style OUT fill:#22c55e,stroke:#16a34a,color:#fff

This diagram shows where NAT fits in the routing pipeline. An incoming packet first passes through pre-routing DNAT, where the destination address is translated before the routing decision. Then the routing decision occurs - the forwarding table lookup uses the translated destination to determine the outgoing interface. Finally, post-routing SNAT translates the source address after routing. The order matters: DNAT happens before routing so the routing decision uses the new destination, and SNAT happens after routing so the routing decision uses the original destination.

Hairpin NAT

Hairpin NAT (also called NAT loopback or NAT reflection) is a scenario where traffic from an internal device to the router's public IP must be routed back to another internal device - the traffic "hairpins" at the NAT device, going out and coming right back in.

The Problem

Imagine a web server at 192.168.1.10 behind a NAT router. The router has a port forwarding rule: public IP 203.0.113.1:443 → 192.168.1.10:443. External clients connect to 203.0.113.1:443 and everything works.

Now an internal client at 192.168.1.20 tries to connect to 203.0.113.1:443 (the same public IP). Without hairpin NAT, this fails. The packet from 192.168.1.20 arrives at the NAT router with destination 203.0.113.1. DNAT translates the destination to 192.168.1.10. The packet is forwarded to the web server. The web server sees a packet from 192.168.1.20 and replies directly to 192.168.1.20 - without going through the NAT router. The internal client receives a response from 192.168.1.10, but it was expecting a response from 203.0.113.1. The TCP handshake fails because the response comes from an unexpected address.

The Solution

Hairpin NAT applies both DNAT (translate destination to internal server IP) and SNAT (translate source to the NAT router's internal IP) for internal-to-internal traffic through the public address. This forces the response to travel back through the NAT router, where the translations are reversed, making the connection work transparently.

Many home routers do not support hairpin NAT. The practical workaround is to use the internal IP address (192.168.1.10) instead of the public address (203.0.113.1) when connecting from within the same network. Split-horizon DNS - returning different IP addresses for the same hostname depending on whether the query comes from inside or outside the network - automates this workaround.

Hairpin NAT in Cloud Environments

Hairpin NAT scenarios also arise in cloud environments. In AWS, if an EC2 instance in a private subnet tries to access a service using the service's public Elastic IP (rather than its private IP), the traffic must leave the VPC through the internet gateway, hit the Elastic IP, and come back into the VPC - a hairpin through the cloud provider's edge infrastructure. AWS handles this transparently, but it adds latency compared to using the private IP directly.

This is why service discovery mechanisms like DNS-based discovery (Route 53 private hosted zones), Kubernetes service DNS, and Consul are important - they enable internal services to find each other using private addresses, avoiding unnecessary hairpin paths through NAT and public internet infrastructure. The general principle: internal communication should stay internal, using private addresses resolved through internal DNS, rather than bouncing through public NAT boundaries.

Hairpin NAT in Kubernetes

Kubernetes has its own hairpin NAT considerations. When a pod sends traffic to its own service's ClusterIP, the traffic is DNAT'd to one of the backing pods - which might be the same pod that sent the request. kube-proxy handles this by enabling hairpin mode on the bridge interface, allowing packets to be forwarded back to the same interface they arrived on. Without this, self-referencing service traffic fails silently.

NAT and Firewall Interaction

NAT and firewalls are distinct functions, but they often run on the same device and interact closely. Understanding the order of operations is critical for writing correct firewall rules:

Inbound traffic: DNAT occurs before firewall filtering. The firewall sees the translated (internal) destination address, not the original public address. If you want to allow traffic to your web server at 192.168.1.10:443, your firewall rule must reference 192.168.1.10:443 - not the public IP.

Outbound traffic: Firewall filtering occurs before SNAT. The firewall sees the original internal source address, not the translated public address. If you want to allow 192.168.1.10 to access the internet, your rule must reference 192.168.1.10 - the SNAT translation happens after the rule is evaluated.

On Linux, this is implemented in the netfilter/iptables (or nftables) framework, which processes packets through a series of chains:

Incoming: PREROUTING (DNAT) → FORWARD (filter) → POSTROUTING (SNAT)
Outgoing: OUTPUT (filter) → POSTROUTING (SNAT)

Understanding this pipeline is essential for writing iptables rules correctly - a common task for system administrators and DevOps engineers managing Linux-based firewalls and NAT gateways.

NAT State Tracking and Connection Tables

NAT is not a stateless operation. Every NAT translation creates an entry in a connection tracking table (conntrack in Linux) that maps the original packet headers to the translated headers. This state must be maintained for the duration of the connection - otherwise, return packets cannot be translated back correctly.

Connection Tracking in Linux

Linux's conntrack system tracks every connection passing through the NAT device. Each entry records:

The conntrack table has a configurable maximum size (default 65,536 entries on many Linux distributions, tunable via nf_conntrack_max). When this table fills up, new connections are dropped - a common cause of mysterious connectivity failures in high-traffic NAT environments.

Symptoms of conntrack exhaustion: Intermittent connection failures that affect new connections while existing connections continue working. The kernel logs the message nf_conntrack: table full, dropping packet. This problem frequently occurs on Kubernetes nodes handling thousands of concurrent connections, NAT gateways for busy microservice clusters, and Docker hosts running many containers.

Mitigation strategies include: increasing nf_conntrack_max, reducing conntrack timeouts for protocols that do not need long timeouts (UDP default timeout is 30 seconds, but connections to DNS servers need only a few seconds), and using eBPF-based connection tracking that bypasses the conntrack subsystem entirely for known high-volume traffic patterns.

NAT Timeout Implications

Different protocols have different NAT timeout behaviors that affect application design:

TCP: NAT entries for TCP connections typically remain for 5 days (Linux default: 432,000 seconds for established connections). TCP's explicit connection teardown (FIN/RST) triggers immediate removal of the NAT entry.

UDP: NAT entries for UDP are much shorter (Linux default: 30 seconds for unreplied, 120 seconds for assured). Because UDP has no connection state, the NAT device cannot know when a UDP "connection" is finished - it relies entirely on timeouts. Applications using UDP for long-lived communication (VoIP, gaming, DNS over UDP with keep-alive) must send periodic packets to prevent NAT timeout.

ICMP: NAT entries for ICMP are very short (default 30 seconds). This is usually sufficient since ICMP is used for transient diagnostics, not sustained communication.

NAT in Microservice and Container Networking

Modern microservice architectures use NAT extensively, often in ways that are hidden by the container orchestration platform:

Docker NAT

When a Docker container accesses the internet, Docker performs SNAT (IP masquerading) on the container's traffic. The container has a private IP on the Docker bridge network (typically 172.17.0.x). Outbound traffic is translated to the host machine's IP before leaving. Inbound traffic uses port forwarding (DNAT) - docker run -p 8080:80 creates a DNAT rule that forwards host port 8080 to container port 80.

Kubernetes NAT

Kubernetes networking involves multiple NAT layers depending on the CNI plugin and service type:

ClusterIP services use DNAT. Traffic to the service's virtual IP is translated (by kube-proxy or eBPF) to one of the backing pod IPs. This translation happens on every node and is the foundation of Kubernetes service discovery.

NodePort services add another DNAT layer: traffic arriving at any node's IP on the NodePort is translated to the service's ClusterIP, which is then translated again to a pod IP. Double DNAT.

Pod-to-external traffic typically uses SNAT (IP masquerading) to translate the pod's cluster-internal IP to the node's IP when traffic leaves the cluster. Without this SNAT, external servers would receive traffic from pod IPs that they cannot route responses back to.

LoadBalancer services in cloud environments often use the cloud provider's NAT infrastructure. AWS Network Load Balancers can operate in "client IP preservation" mode (no SNAT, the backend sees the client's real IP) or "proxy protocol" mode, depending on the configuration.

NAT and Observability

NAT creates a significant observability challenge: the source IP that appears in server logs may not be the actual client IP. When traffic passes through a load balancer with SNAT, all requests appear to come from the load balancer's IP. This makes it impossible to identify individual clients from server logs alone.

Solutions include the X-Forwarded-For HTTP header (which carries the original client IP through HTTP-layer proxies), the PROXY protocol (which carries original addresses at the TCP layer), and "client IP preservation" features in cloud load balancers that avoid SNAT entirely by using DSR (Direct Server Return) or similar techniques.

graph TD
    subgraph K8sNAT["Kubernetes Service NAT Layers"]
        direction TB
        CLIENT["External Client<br/>203.0.113.50"] -->|"Dst: NodeIP:30080"| NODE["Node<br/>10.0.1.5"]
        NODE -->|"DNAT: NodePort → ClusterIP<br/>Dst: 10.96.0.1:80"| KPROXY["kube-proxy<br/>(iptables/eBPF)"]
        KPROXY -->|"DNAT: ClusterIP → PodIP<br/>Dst: 10.244.1.15:80<br/>SNAT: Src → NodeIP"| POD["Pod<br/>10.244.1.15"]
    end

    style CLIENT fill:#4f46e5,stroke:#4338ca,color:#fff
    style NODE fill:#7c3aed,stroke:#6d28d9,color:#fff
    style KPROXY fill:#8b5cf6,stroke:#7c3aed,color:#fff
    style POD fill:#22c55e,stroke:#16a34a,color:#fff

This diagram shows the NAT layers in Kubernetes service routing. An external client sends traffic to a node's IP on a NodePort. The node performs DNAT to translate the destination from the NodePort to the ClusterIP service address. kube-proxy then performs a second DNAT to translate the ClusterIP to a specific pod IP, and also performs SNAT to change the source to the node's IP so the pod's response routes back through the same node for proper reverse translation.

Interview: NAT in the routing context appears in cloud architecture and Kubernetes discussions:

"Why can't your backend see the client's real IP?" - SNAT at the load balancer or NAT gateway replaced the client's source IP with the load balancer's IP. Use X-Forwarded-For headers, PROXY protocol, or client IP preservation features to recover the original IP.

"How does a Kubernetes ClusterIP service work?" - It is DNAT. kube-proxy (or eBPF in newer implementations) installs iptables/nftables rules that translate the service's virtual IP to a randomly selected backing pod IP. The pod never sees the ClusterIP - it receives traffic with its own IP as the destination.

"Traffic from pods to an external API is being blocked. Why?" - The external API's firewall may be filtering by source IP. Pod traffic is typically SNAT'd to the node's IP. If you have multiple nodes, the external API's allowlist needs every node's IP (or the NAT Gateway's IP if using one). This is why many teams route all egress through a single NAT Gateway with a known Elastic IP.

Module Summary

This module covered the routing and forwarding mechanisms that move packets across the internet. You now understand how routers make forwarding decisions, the difference between static and dynamic routing, the evolution from distance vector (RIP) to link-state (OSPF) protocols, how BGP connects autonomous systems into the global internet, the constraints imposed by MTU and fragmentation, and how NAT interacts with routing in modern cloud and container environments. These concepts are the foundation for understanding network design decisions in System Design interviews.

Exercise: Starter: If you run Docker locally, start a container and examine the NAT rules Docker creates. Run sudo iptables -t nat -L -n -v and look for rules in the DOCKER chain. Can you identify the DNAT rules for port-forwarded containers and the MASQUERADE (SNAT) rules for outbound container traffic?

Intermediate: Research how kube-proxy implements Kubernetes services in iptables mode. Run sudo iptables -t nat -L -n on a Kubernetes node and find the KUBE-SERVICES and KUBE-SEP chains. Trace the DNAT rules from service ClusterIP to individual pod IPs. How does kube-proxy implement random load balancing across pods using iptables probability matching?

Advanced: Compare kube-proxy's iptables mode with eBPF-based service routing (as implemented by Cilium). How does eBPF avoid the O(n) iptables rule traversal problem that affects large Kubernetes clusters with thousands of services? What performance improvements does eBPF provide over iptables-based NAT? Research the concept of "conntrack" (connection tracking) in Linux - how does it maintain NAT state, and what happens when the conntrack table fills up in a high-traffic environment?

Back to Computer Networking