AlgoInsight

System Design

Load Balancer vs Reverse Proxy vs API Gateway

Three components that all sit in front of your servers, all intercept traffic, and all get confused for one another. Here's what each actually does, how they overlap, and how to tell them apart.

By Admin2026-06-1813 min readload-balancerreverse-proxyapi-gatewaysystem-designarchitecturenetworking

These three terms cause more confusion than almost anything else in system design - and for good reason. A load balancer, a reverse proxy, and an API gateway all sit in front of your backend servers, all intercept incoming traffic, and all forward it on. Their roles overlap heavily, and in practice a single piece of software often plays more than one of these roles at once.

But they aren't the same thing, and understanding the distinction is a genuine mark of system design maturity. The cleanest way to think about it: they sit on a spectrum of increasing application-awareness. A reverse proxy is the general concept; a load balancer is a reverse proxy focused on distributing load; an API gateway is a reverse proxy packed with application-level features. Let's build that understanding from the ground up.

graph LR
    C[Clients] --> X[Front-door component]
    X -.reverse proxy: forwards & shields.-> S[Backend servers]
    X -.load balancer: distributes load.-> S
    X -.API gateway: app-aware routing,<br/>auth, rate limiting, aggregation.-> S

This diagram shows clients sending requests to a front-door component that forwards them to backend servers. The same position can be filled by three overlapping roles: a reverse proxy, which forwards and shields; a load balancer, which distributes load across servers; or an API gateway, which adds application-aware routing, authentication, rate limiting, and aggregation.

Reverse proxy: the general concept

Start here, because it's the umbrella the other two fall under. A reverse proxy is a server that sits in front of one or more backend servers and forwards client requests to them. To the client, the reverse proxy is the server - clients never talk to the backends directly; they only ever see the proxy.

(The "reverse" distinguishes it from a forward proxy, which sits in front of clients and forwards their requests out to the internet - the kind a company might use to filter employee web access. A reverse proxy is the opposite: it sits in front of servers, representing them to the outside world.)

A reverse proxy provides several benefits just by being that intermediary:

The crucial point: reverse proxy is the broad category. A load balancer and an API gateway are both, fundamentally, specialized reverse proxies. Once you see that, the relationship between the three becomes much clearer.

Load balancer: the reverse proxy focused on distribution

A load balancer is a reverse proxy whose primary job is to distribute incoming requests across multiple backend servers so that no single server is overwhelmed. When you run many copies of a service to handle traffic, the load balancer is what decides which copy each request goes to.

Its core purposes:

A load balancer can operate at different levels of the network. A Layer 4 (transport-level) load balancer routes based on network information like IP addresses and ports, without looking at the actual content of requests - it's fast and simple. A Layer 7 (application-level) load balancer looks at the content of HTTP requests and can route based on the URL path, headers, or cookies - more flexible, but with a bit more overhead. This distinction matters because Layer 7 load balancing starts to blur into what an API gateway does.

The defining trait of a load balancer is its focus on distribution and availability. It's less concerned with the meaning of requests than with spreading them efficiently and routing around failure.

API gateway: the reverse proxy packed with features

An API gateway is a reverse proxy that adds a rich set of application-level features on top of forwarding requests. It's the most feature-heavy and application-aware of the three. Where a load balancer asks "which server should handle this?", an API gateway asks deeper questions: "Is this caller authenticated? Are they within their rate limit? Which service does this path map to? Should I combine responses from several services?"

Its responsibilities typically include request routing based on application logic, authentication and authorization, rate limiting, response aggregation across multiple services, protocol translation, and centralized logging - the full set covered in our dedicated API gateway guide.

The defining trait of an API gateway is application awareness. It understands your API - its endpoints, its authentication scheme, its services - and makes decisions based on that understanding, not just on network-level distribution. It's the front door specifically for an API, especially in a microservices architecture.

The spectrum: how they relate

Here's the mental model that ties it all together. These three aren't three unrelated things - they're points on a spectrum of increasing application-awareness:

Reverse Proxy Load Balancer API Gateway
Core role Forward & shield Distribute load Application-aware front door
Primary concern Intermediary, security Availability, distribution API logic, cross-cutting concerns
Awareness level Network/HTTP Network (L4) or HTTP (L7) Deep application understanding
Typical features Caching, SSL, hiding backends Health checks, load algorithms Auth, rate limiting, aggregation, routing
Where it shines Any backend that needs a front Many copies of a service Microservices, public APIs

Read down the columns and you can see the progression: a reverse proxy is the foundational concept (forward and shield); a load balancer adds the specific intelligence of distribution and health-aware routing; an API gateway adds full application-level understanding. Each is "a reverse proxy plus more," with the API gateway being the most specialized.

Why a single tool often plays all three roles

In practice, the lines blur because the same software frequently fills more than one role. Popular tools like NGINX can act as a reverse proxy, a load balancer, and an API gateway depending on how you configure them - and many real deployments use them in all three capacities at once. A cloud load balancer might do health checks (load balancer), terminate SSL (reverse proxy), and route by path (edging into gateway territory).

This is exactly why the terms get conflated: the roles are distinct, but the tools that perform them overlap. The right way to think about it is to focus on the role a component is playing in a given moment, not on which product it happens to be. A component "is acting as a load balancer" when it's distributing requests across servers; the same component "is acting as an API gateway" when it's checking authentication and routing by application logic.

Tip: Don't get hung up on labels. Ask what function is needed at a given point in the architecture. Do you need to spread traffic across identical servers? That's load balancing. Do you need a single front door that authenticates and routes to many different services? That's a gateway. Do you just need to shield and front a backend? That's a reverse proxy. The function determines the answer, even if one tool provides all three.

How they work together in a real system

These components aren't either-or - a real architecture often uses several together, layered. A common arrangement in a large system:

  1. Traffic first hits a load balancer that distributes it across multiple instances of the API gateway (so the gateway itself isn't a single point of failure).
  2. The API gateway authenticates the request, enforces rate limits, and routes it to the appropriate backend service.
  3. In front of each backend service, another load balancer (or the gateway's own load-balancing) distributes the request across that service's instances.

So a single request might pass through a load balancer, then a gateway, then another load balancer before reaching a server. Each layer does its specific job. Understanding that these compose - rather than compete - is the key insight.

graph LR
    C[Clients] --> LB1[Load Balancer]
    LB1 --> GW1[API Gateway] & GW2[API Gateway]
    GW1 --> LB2[LB] --> SA1[Service A] & SA2[Service A]
    GW1 --> LB3[LB] --> SB1[Service B] & SB2[Service B]

This diagram shows the three components working together in layers. Clients hit a load balancer, which distributes traffic across multiple API gateway instances so the gateway isn't a single point of failure. Each gateway routes requests to backend services through additional load balancers that spread the load across multiple instances of Service A and Service B.

Choosing in an interview

Interview: If an interviewer asks you to distinguish these, the strongest answer leads with the spectrum: "They overlap because a load balancer and an API gateway are both specialized reverse proxies. A reverse proxy is the general front-door concept; a load balancer focuses on distributing traffic across servers for availability; an API gateway adds application-level features like authentication, rate limiting, and routing for microservices." Then note that one tool often plays all three roles, so what matters is the function needed at each point. That framing shows you understand the relationship, not just three separate definitions.

Forward proxy vs reverse proxy: don't confuse them

Since "proxy" is central to this whole topic, it's worth nailing down a distinction people often blur: forward proxy versus reverse proxy. They sound similar and are opposites.

A forward proxy sits in front of clients and represents them to the outside world. When employees in a company browse the web through a corporate proxy that filters and logs their traffic, that's a forward proxy - it acts on behalf of the clients, and the destination servers see the proxy instead of the individual users.

A reverse proxy sits in front of servers and represents them to the outside world. Clients think they're talking to the server, but they're actually talking to the reverse proxy, which forwards their requests to the real backends behind it. Load balancers and API gateways are both reverse proxies - they front servers, not clients.

The simple memory aid: a forward proxy hides the client from the server; a reverse proxy hides the server from the client. Everything in this article concerns reverse proxies.

Real-world examples of each role

Concrete examples make the roles click:

Notice that NGINX appears in the first two and could appear in the third - the same tool, different roles, depending on configuration. That's the whole reason these terms blur in practice.

A quick decision guide

When you're deciding what you need at a given point in an architecture:

And remember these compose: a real system often has a load balancer in front of redundant gateways, with more load balancing between the gateway and each service's instances. You're rarely choosing one - you're deciding which roles you need and how to layer them.

In cloud environments, all three roles are usually available as managed services - managed load balancers, managed reverse proxies (often part of CDN offerings), and managed API gateways - each handling its own scaling and redundancy for you. This makes it easy to compose them without operating the infrastructure yourself, but it doesn't change the underlying concepts: you're still choosing which role each managed service plays in your request path.

Why the confusion is so persistent

It's worth naming why these three are conflated more than almost any other trio in system design. Three reasons compound. First, they occupy the same position in the architecture - between clients and backends - so diagrams draw them in the same spot. Second, their features genuinely overlap: a Layer 7 load balancer routing by URL path is doing something an API gateway also does; a reverse proxy that distributes requests is load balancing. Third, and most importantly, the tools deliberately blur the lines - products like NGINX and cloud services are built to play whichever role you configure, so the same product legitimately is all three at different moments.

The takeaway isn't to memorize rigid definitions and police them. It's to recognize that "reverse proxy," "load balancer," and "API gateway" name roles a component plays, not mutually exclusive product categories. When you read a system diagram, ask what function each box performs rather than fixating on its label - and when you design one, choose based on the functions you need. That functional thinking is what the distinction is really teaching.

The bottom line

A reverse proxy is the umbrella concept: an intermediary that fronts your servers. A load balancer is a reverse proxy specialized for distributing load and maintaining availability. An API gateway is a reverse proxy specialized for application-level concerns in a multi-service system. They sit on a spectrum of increasing application-awareness, they frequently coexist in layers within one architecture, and one tool can play all three roles at once.

The confusion around these terms melts away once you stop asking "which one is this?" and start asking "which role is being played here?" That shift - from labels to functions - is exactly the kind of clear thinking that system design rewards. We build that clarity across System Design Fundamentals.

Go deeper

More articles