Free lesson · System Design Crash Course

What a Reverse Proxy Is and Why

Medium11 min readsystem-designhlddistributed-systems

Lesson 6.1 - What a Reverse Proxy Is and Why

The first lesson of Module 6.

Let me start with a story. You're building a small web app. You have one server running your code, and one IP address pointing at that server. Users type your URL, requests hit that IP, your server responds. Everything works.

Then your app gets popular. Traffic grows. The one server can't handle it. You provision a second server, then a third. Now you have three servers - but only one IP address. How do users' requests get distributed across the three servers? They can't all hit just one server; that defeats the purpose of having three.

This is the problem a reverse proxy solves. A reverse proxy is a server that sits in front of your other servers, receives all incoming requests, and forwards each to one of the backend servers behind it. Users see one IP, one URL, one "thing." Behind that single entry point, the proxy maintains a pool of real servers and distributes work among them.

This pattern is the cornerstone of how modern web infrastructure scales. Almost every production system at any meaningful scale uses one or more reverse proxies. This lesson covers what they are, why they're called "reverse" (versus "forward"), and what they actually do beyond just forwarding.

The Core Pattern

Imagine the architecture visually:

flowchart LR
    U1["User 1"]
    U2["User 2"]
    U3["User 3"]
    LB["Reverse Proxy<br/>(load balancer)"]
    S1["App Server 1"]
    S2["App Server 2"]
    S3["App Server 3"]

    U1 --> LB
    U2 --> LB
    U3 --> LB
    LB --> S1
    LB --> S2
    LB --> S3

    style LB fill:#fbbf24,stroke:#f59e0b,color:#000
    style S1 fill:#34d399,stroke:#10b981,color:#000
    style S2 fill:#34d399,stroke:#10b981,color:#000
    style S3 fill:#34d399,stroke:#10b981,color:#000

A diagram showing the reverse proxy pattern. Three users on the left send requests to a single endpoint  - the reverse proxy in the middle. The reverse proxy receives all incoming traffic and forwards each request to one of three app servers behind it. From the user's perspective, there's only one server  - the IP and URL they see is the reverse proxy. From the application servers' perspective, the proxy is acting like a client  - it's the one making requests to them. The reverse proxy mediates everything between the public internet and the application infrastructure. This pattern is essentially universal in production systems above a certain size; it's how single-server systems graduate to multi-server systems without breaking client expectations.

The proxy is the single externally-visible entity. The backend servers are private - clients never connect to them directly. The proxy chooses which backend handles each request based on whatever algorithm it's configured to use (round robin, least connections, etc., which we'll cover in Lesson 6.3).

Notice some properties of this design:

1. One IP, many servers. Users connect to one place; the proxy distributes the work.

2. Backends are anonymous. Users have no idea if there's 1 backend or 100. They can't tell when a backend dies; they can't tell when a new one comes online.

3. The proxy is a single point of failure - unless you also have multiple proxies (you usually do, fronted by DNS or another layer).

4. The proxy can do things beyond forwarding. TLS termination, caching, request filtering, rate limiting, logging - many features live in the proxy.

This pattern is universal in modern architectures. It's worth internalizing deeply.

Forward Proxy vs Reverse Proxy

The naming convention is confusing for newcomers. Let me make it clear.

A forward proxy sits between a client and the internet. Clients configure their browsers to send traffic through it; the proxy fetches resources on the client's behalf and returns them. Forward proxies are used for:

Corporate networks - proxies filter and log employee traffic
Privacy tools - proxies hide a user's real IP from destination sites
Content filtering - schools and libraries block certain sites
Bypassing geographic restrictions - VPNs work similarly

A reverse proxy sits between the internet and a server (or pool of servers). Clients don't know it exists - they just send requests to what looks like the server, and the proxy decides what to do with each one.

flowchart LR
    subgraph Forward["Forward Proxy"]
        U1["User<br/>(behind proxy)"]
        FP["Forward Proxy"]
        Internet1["Internet"]
        U1 --> FP --> Internet1
    end

    subgraph Reverse["Reverse Proxy"]
        Internet2["Internet (users)"]
        RP["Reverse Proxy"]
        S["Backend servers"]
        Internet2 --> RP --> S
    end

    style FP fill:#a78bfa,stroke:#8b5cf6,color:#000
    style RP fill:#fbbf24,stroke:#f59e0b,color:#000

A diagram comparing forward and reverse proxies. In the forward proxy case on the left, the user is behind the proxy. The user's traffic goes through the proxy first, then out to the broader internet. The proxy works on behalf of the user. Common use cases: corporate networks filtering employee traffic, VPNs hiding identity, schools blocking sites. In the reverse proxy case on the right, the user is in the internet. Their traffic hits the reverse proxy at the edge of someone's infrastructure. The proxy then forwards the request to backend servers. The proxy works on behalf of the servers. Common use cases: load balancing, TLS termination, caching, CDN edges. The "forward" versus "reverse" naming refers to whose side the proxy is on. Both patterns exist; both are common; but they solve very different problems.

A useful mental shortcut: a forward proxy is on the client side; a reverse proxy is on the server side. When we say "load balancer," "API gateway," "Nginx," "Cloudflare in front of your site" - we mean reverse proxies. Forward proxies don't come up much in HLD for typical web services.

For the rest of this module (and for most of system design), when we say "proxy," we mean reverse proxy unless otherwise specified.

Why Reverse Proxies Exist: The Full Function Set

Reverse proxies started as load balancers - distributing traffic across multiple backends. But they accumulated many other functions over the years. Modern reverse proxies often do all of these:

1. Load balancing. The core function. Distribute traffic across backend servers.

2. TLS termination. Decrypt HTTPS traffic at the proxy, send unencrypted (or simply-encrypted) traffic to backends. Concentrates the CPU-intensive crypto work.

3. Caching. Cache responses to common requests at the proxy, reducing load on backends. Especially for static content or cacheable API responses.

4. Compression. Gzip or Brotli responses before sending to clients. Reduces bandwidth.

5. Request routing. Look at the URL or headers to decide which backend pool handles each request. /api/* to one pool, /static/* to another.

6. Rate limiting. Reject or slow requests from clients that exceed thresholds. Protects backends from abuse.

7. Authentication offload. Validate JWTs or other tokens before forwarding. Backends can trust the proxy did the auth work.

8. Header manipulation. Add X-Forwarded-For, strip auth headers, normalize headers.

9. Security filtering. Block known bad IPs, filter malicious requests, do WAF (Web Application Firewall) work.

10. Health checking. Periodically test backends; route around dead ones automatically.

11. Connection pooling. Multiplex many client requests over fewer backend connections.

12. Request buffering. Hold slow clients' requests until complete before sending to backend; protects backends from slow-client attacks.

A modern Nginx config might do half of these in one box. AWS ALB does most of them as a managed service. The proxy isn't just routing - it's the multi-tool that mediates between client and server, doing all the cross-cutting concerns that would otherwise need to live in each backend.

A Mental Model: The Front Desk

Here's an analogy I find useful. Imagine a hotel:

The lobby and front desk are like the reverse proxy
The individual rooms are like the backend servers
The guests are like users

When a guest arrives, they don't wander around looking for an open room. They go to the front desk. The front desk:

Knows which rooms are occupied
Knows which rooms are clean / under repair / available
Decides which room to assign based on the guest's needs
Handles all the access control (key cards)
Knows the language of the guests (the protocol they speak)
Handles things like printing receipts and answering questions
Logs every check-in for the auditors

The rooms themselves are simpler. They don't need to know how to greet guests. They don't need to know the language. They just need to be ready when someone arrives.

A reverse proxy works exactly like this for HTTP requests. The proxy is sophisticated and busy; the backends behind it can be simple, focused on doing one thing well - serving the application.

This separation is the architectural win. Without the proxy, every backend would need its own TLS handling, its own rate limiting, its own routing logic. With the proxy, these concerns live in one place and backends stay simple.

A Day in the Life of a Request

Let me trace one HTTPS request through a typical reverse proxy setup.

User types URL. Browser does DNS lookup, gets IP of the reverse proxy.
TCP+TLS handshake to proxy. Browser establishes encrypted connection with the proxy.
HTTP request to proxy. Browser sends GET /api/users HTTP/2.
Proxy receives, decrypts. It now has the plaintext HTTP request.
Proxy applies rules. Is this request rate-limited? Authorized? Cacheable? Suppose all checks pass.
Proxy chooses backend. Using the configured algorithm (say, round robin), picks Backend 2.
Proxy forwards request. Sends a (possibly modified) HTTP request to Backend 2. Adds X-Forwarded-For header with the original client IP.
Backend processes. Backend 2 receives the request as if from any client, runs application logic, returns a response.
Proxy receives response. May modify (compress, add headers), may cache.
Proxy sends response to client. Encrypted via the TLS connection from step 2.
Client receives, displays.

sequenceDiagram
    participant C as Client
    participant P as Reverse Proxy
    participant B as Backend Server

    C->>P: TCP+TLS handshake
    C->>P: HTTPS request (encrypted)
    Note over P: 1. Decrypt<br/>2. Apply rules (rate limit, auth)<br/>3. Choose backend (round robin)<br/>4. Add X-Forwarded-For
    P->>B: HTTP request (often plaintext)
    Note over B: Process<br/>(run app logic)
    B->>P: HTTP response
    Note over P: 5. Compress<br/>6. Maybe cache<br/>7. Re-encrypt
    P->>C: HTTPS response

A sequence diagram showing one request flowing through a reverse proxy. The client does the TCP and TLS handshake with the proxy and sends an HTTPS request. The proxy decrypts the request, applies its rules  - rate limiting, authentication, anything else  - then chooses a backend server using its load balancing algorithm. It adds an X-Forwarded-For header so the backend knows the original client IP, and forwards the request, often unencrypted to a backend on a trusted internal network. The backend processes the request and returns a response. The proxy may compress the response, may cache it, then re-encrypts and sends back to the client. The backend never saw the original encrypted connection; the client never knows the request was forwarded. Everyone gets what they want without knowing the full picture. This indirection is the proxy's whole purpose.

This is the basic flow. Variations include caching at step 5 (return cached response without going to backend), routing different paths to different backend pools, and so on. The core pattern stays the same.

What Backends See

An important consequence of the proxy pattern: backends see requests from the proxy, not from the client. Without special handling, the backend's view of "who is the client" is wrong:

The TCP connection is from the proxy's IP, not the user's
The TLS session (if any) is between the proxy and backend, not user and backend
The Host header might be the internal hostname, not the public URL

To preserve the original information, proxies add headers:

X-Forwarded-For - the original client IP (or a chain if multiple proxies)
X-Forwarded-Proto - was the original protocol HTTP or HTTPS?
X-Forwarded-Host - original Host header
X-Real-IP - sometimes used as an alternative to X-Forwarded-For

A subtle but important point: these headers can be spoofed by clients. A direct request from a client could include a fake X-Forwarded-For header. The backend must only trust these headers if the request came from a known proxy. This is a common security pitfall - see lots of "IP-based authentication bypassed via X-Forwarded-For spoofing" CVEs.

The fix: configure your proxy to strip incoming X-Forwarded-* headers from external requests and add its own. Backends should only see and trust the headers added by the proxy. This is part of the standard Nginx/HAProxy/Envoy configuration in production.

Variations and Topologies

A typical real-world deployment has more than one layer of proxies:

flowchart TD
    Users["Users"]
    CDN["CDN / Edge<br/>(Cloudflare, Fastly)"]
    LB["Public Load Balancer<br/>(ALB, Nginx)"]
    IP["Internal Proxy / API Gateway<br/>(Envoy, Nginx)"]
    S1["Service A"]
    S2["Service B"]
    S3["Service C"]

    Users --> CDN
    CDN --> LB
    LB --> IP
    IP --> S1
    IP --> S2
    IP --> S3

    style CDN fill:#a78bfa,stroke:#8b5cf6,color:#000
    style LB fill:#fbbf24,stroke:#f59e0b,color:#000
    style IP fill:#fbbf24,stroke:#f59e0b,color:#000

A diagram showing a typical multi-layer proxy topology in modern infrastructure. At the top, users send requests to a CDN or edge network like Cloudflare or Fastly. The CDN handles caching of static content, DDoS protection, and routing to the nearest data center. From the CDN, requests flow to a public load balancer, perhaps AWS ALB or Nginx, which handles TLS termination and distributes to data center resources. From there, requests reach an internal proxy or API gateway  - often Envoy or Nginx  - that routes between microservices, handles internal authentication, and applies rate limiting. Finally requests land on the actual application services. Each layer adds value: edge does global routing and caching; public LB does TLS and traffic shaping; internal proxy does service routing and observability. The cost is added latency, but each layer is doing real work. Most large production systems have 3 to 5 layers of proxies in front of the actual application code.

Each layer has a different role:

CDN handles caching, geographic routing, DDoS protection
Public load balancer handles TLS termination, traffic distribution to a data center
Internal proxy handles service-to-service routing, internal observability

The user's request might pass through all three. Each layer adds a small amount of latency but provides real functionality. The aggregate latency cost is usually a few milliseconds - worth it for the capabilities.

A Real-World Story

When I was at Stripe, the customer-facing API had multiple proxy layers between users and our app servers. Cloudflare for DDoS protection and edge caching. Then a layer of Nginx for TLS termination and basic routing. Then envoy for service mesh routing. Then finally the actual Stripe service.

At one point, we had an issue where a class of requests was timing out for a specific customer. Investigation revealed: the issue was at a specific proxy layer's idle timeout. Their requests took longer than the proxy's max-idle-time, so the proxy was killing the connection before the response came back. Fixing it required reading three different proxy configurations to find which one was responsible.

This is the operational reality of multi-layer proxy setups. Each layer has its own timeouts, its own retry behavior, its own logging, its own configuration model. When something goes wrong, you might need to dig through all of them to find the cause. The trade-off for all the features they provide.

Knowing this from day one - and being deliberate about the layers in your architecture - makes operations much easier.

Six Takeaways from This Lesson

A reverse proxy sits in front of backend servers, mediates traffic, and presents a single entry point to the world.
Forward proxies are on the client side; reverse proxies are on the server side. When we say "load balancer" or "API gateway," we mean reverse proxy.
Reverse proxies do many things beyond load balancing - TLS termination, caching, routing, rate limiting, security, header manipulation. They're multi-tools.
Backends see traffic from the proxy, not the client. Use X-Forwarded-* headers (carefully) to preserve original information.
Multi-layer proxy topologies are normal at scale - CDN → public LB → internal proxy → app. Each layer adds value.
The mediation pattern is fundamental to modern architecture. Almost every meaningful system uses one or more reverse proxies. Internalize the pattern.

Try It

Exercise: Warmup (10 minutes). For a website you use, use curl -I to fetch the headers. Look for Server: header - it often reveals the reverse proxy (Nginx, Cloudflare, etc.). Some sites hide this.

Exercise: Core (20 minutes). Run Nginx locally with a simple config that proxies to two backend servers (you can use simple Python HTTP servers). Watch how requests get distributed.

Exercise: Stretch (40 minutes). Read about a major outage caused by a reverse proxy misconfiguration (Cloudflare, Fastly, AWS ALB have all had public ones). Note how a single layer can take down massive amounts of internet.

What's Next

Lesson 6.2 covers Layer 4 vs Layer 7 load balancing - the fundamental architectural distinction in how proxies route traffic. Different layers; different capabilities; different costs.

Back to System Design Crash Course