Free lesson · System Design Advanced

Edge Architecture Patterns

Hard12 min readsystem-designadvanced

A user in São Paulo clicks a button on your web app. That click becomes an HTTP request that travels through undersea fibre cables to your API server in us-east-1 (Virginia) - a round trip of about 160 milliseconds just for the network. Your server processes the request in 20ms, queries a database in 5ms, and sends the response back. Total time: 185ms. The user barely notices.

Now imagine that click triggers three sequential API calls - an auth check, a data fetch, and a personalisation call. Each one makes that 160ms round trip. Three times 185ms is 555ms. The user notices. They feel the lag. On mobile with higher latency, it's worse.

Edge computing flips this model. Instead of sending every request to a central data centre, you push computation to servers that are geographically close to the user. A Cloudflare Worker running in São Paulo handles that request in 5ms of network latency instead of 160ms. The user's click feels instant.

But "just run code at the edge" sounds simpler than it is. Edge environments have severe constraints - limited CPU time, no persistent local storage, no access to your primary database, restricted runtime APIs. Understanding what can and what should run at the edge versus what belongs in your origin data centre is the architectural decision this lesson covers.

What "The Edge" Actually Means

graph TD
    subgraph "The Network Hierarchy"
        USER["User's device<br/>(browser, mobile app)"]
        
        USER --> POP["Edge POP (Point of Presence)<br/>──────────────────────<br/>CDN server in user's city<br/>or nearest metro area<br/><br/>100-300 global locations<br/>Latency to user: 5-20ms<br/><br/>Traditionally: static file caching<br/>Now: compute (Workers, Lambda@Edge)"]
        
        POP --> REGIONAL["Regional Edge<br/>──────────────────────<br/>AWS CloudFront Regional Edge Cache<br/>or dedicated regional servers<br/><br/>10-30 global locations<br/>Latency to POP: 10-50ms<br/><br/>Larger cache, more compute power"]
        
        REGIONAL --> ORIGIN["Origin Data Centre<br/>──────────────────────<br/>Your primary servers + database<br/><br/>1-3 locations (us-east-1, eu-west-1)<br/>Latency to regional: 50-200ms<br/><br/>Full compute, full data access,<br/>the 'source of truth'"]
    end

This diagram shows the network hierarchy from user to origin. The user's device connects to an edge point of presence (POP) in their city with 5 to 20 milliseconds latency - there are 100 to 300 of these globally. The POP connects to a regional edge with 10 to 50 milliseconds latency - about 10 to 30 locations. The regional edge connects to the origin data centre with 50 to 200 milliseconds latency - typically 1 to 3 locations. Traditionally edge POPs only cached static files, but now they can run compute via platforms like Cloudflare Workers and Lambda@Edge.

The key insight: every tier in this hierarchy trades capability for proximity. The edge POP is close to the user but has limited compute and no database. The origin is far from the user but has everything. The architectural decision is about which work happens where.

The Three Major Edge Compute Platforms

graph TD
    subgraph "Cloudflare Workers"
        CF["Runtime: V8 Isolates<br/>(same engine as Chrome)<br/><br/>Language: JavaScript, TypeScript, Rust, WASM<br/><br/>Cold start: ~0ms (no container to boot)<br/>CPU limit: 10-50ms per request (paid plan)<br/>Memory: 128MB<br/>Locations: 300+ cities<br/><br/>Storage: Workers KV (global eventual),<br/>Durable Objects (strongly consistent),<br/>R2 (S3-compatible object store),<br/>D1 (SQLite at the edge)<br/><br/>Strength: fastest cold start,<br/>largest network, simplest DX"]
    end
    
    subgraph "AWS Lambda@Edge / CloudFront Functions"
        LAMBDA["Runtime: Node.js (Lambda@Edge),<br/>JavaScript (CloudFront Functions)<br/><br/>CloudFront Functions:<br/>• 13 regional locations<br/>• < 1ms cold start<br/>• 2MB code limit, no network access<br/>• For: header manipulation, URL rewrites<br/><br/>Lambda@Edge:<br/>• 13 regional locations<br/>• 1-3 second cold start<br/>• Full Node.js runtime<br/>• Can call origin, APIs, databases<br/>• For: auth, A/B testing, SSR<br/><br/>Strength: deep AWS integration"]
    end
    
    subgraph "Fastly Compute"
        FASTLY["Runtime: WebAssembly (Wasm)<br/><br/>Language: Rust, Go, JavaScript, AssemblyScript<br/><br/>Cold start: ~0ms (Wasm binary)<br/>CPU limit: generous (production workloads)<br/>Memory: configurable<br/>Locations: 80+ POPs<br/><br/>Storage: KV Store,<br/>Config Store, Secret Store<br/><br/>Strength: Wasm performance,<br/>fine-grained cache control"]
    end

This diagram compares three major edge compute platforms. Cloudflare Workers uses V8 isolates with near-zero cold start, supports JavaScript, TypeScript, Rust, and WASM across 300-plus locations, with Workers KV for eventual consistency and Durable Objects for strong consistency. AWS has two tiers: CloudFront Functions for lightweight header manipulation with sub-millisecond cold start, and Lambda@Edge for heavier compute with full Node.js but 1-3 second cold starts. Fastly Compute uses WebAssembly for near-zero cold start with strong Wasm performance and supports Rust, Go, and JavaScript across 80-plus POPs.

Platform Comparison

Important: Cold start matters more than you think. Lambda@Edge's 1-3 second cold start means the first user after a period of inactivity waits seconds for a response. Cloudflare Workers and Fastly Compute use isolate/Wasm models that start in microseconds. For latency-sensitive applications (auth checks, API routing, personalisation), cold start eliminates Lambda@Edge from consideration unless you keep functions warm - which costs money.

What Belongs at the Edge vs Origin

Not everything should run at the edge. The decision framework:

graph TD
    REQUEST["Incoming request"] --> Q1{"Does it need<br/>the primary database?"}
    
    Q1 -->|"Yes, read/write"| ORIGIN["Run at ORIGIN<br/><br/>Examples:<br/>• Order placement<br/>• Account updates<br/>• Complex queries"]
    
    Q1 -->|"No, or read-only cache"| Q2{"Is it latency-<br/>sensitive?"}
    
    Q2 -->|"No"| ORIGIN
    
    Q2 -->|"Yes"| Q3{"Is the compute<br/>lightweight?<br/>(< 50ms CPU)"}
    
    Q3 -->|"No<br/>(heavy ML, batch)"| REGIONAL["Run at REGIONAL<br/><br/>Examples:<br/>• Image resizing<br/>• SSR with data fetch<br/>• ML inference"]
    
    Q3 -->|"Yes"| EDGE["Run at EDGE<br/><br/>Examples:<br/>• Auth token validation<br/>• A/B test routing<br/>• Geo-based redirects<br/>• Header manipulation<br/>• Personalised caching<br/>• Rate limiting<br/>• Bot detection"]

This diagram shows a decision tree for where to run computation. If the request needs the primary database for reads or writes, run at origin. If not, check if it's latency-sensitive. If not latency-sensitive, run at origin. If latency-sensitive, check if the compute is lightweight under 50 milliseconds. If heavy compute like ML inference or image resizing, run at the regional edge. If lightweight, run at the edge POP - this includes auth token validation, A/B test routing, geo-based redirects, rate limiting, and bot detection.

Concrete Edge Use Cases

Tip: The "edge-first" mental model: When designing a request flow, ask "what can I resolve without ever reaching my origin?" Everything you resolve at the edge saves origin compute, reduces latency, and improves the user experience. Auth, routing, caching decisions, redirects, and simple transformations are almost always better at the edge.

Edge State: The Hard Problem

Computation at the edge is straightforward - it's just code running on a server. The hard part is state. Your edge worker validates a JWT - where does it get the signing key? Your edge worker does rate limiting - where does it store the request count? Your edge worker personalises content - where does it read the user's segment?

graph TD
    subgraph "Edge State Options"
        KV["Key-Value Store<br/>(Cloudflare Workers KV, Fastly KV)<br/>──────────────────────<br/>Consistency: Eventual (reads may be stale)<br/>Latency: < 10ms reads globally<br/>Write propagation: 10-60 seconds<br/><br/>Good for: config, feature flags,<br/>A/B test assignments, cached data<br/><br/>Not good for: counters, rate limiting,<br/>anything needing strong consistency"]
        
        DO["Durable Objects<br/>(Cloudflare)<br/>──────────────────────<br/>Consistency: Strong (single-threaded per object)<br/>Latency: depends on object location<br/>Coordination: built-in (each object is a single actor)<br/><br/>Good for: rate limiting, counters,<br/>collaborative state, session state<br/><br/>Trade-off: object is in ONE location,<br/>so users far from it get higher latency"]
        
        CACHE["Edge Cache<br/>(CDN cache with compute-controlled keys)<br/>──────────────────────<br/>Consistency: TTL-based staleness<br/>Latency: < 1ms (local cache hit)<br/>Miss penalty: full round-trip to origin<br/><br/>Good for: API responses, rendered pages,<br/>any content with a TTL > 5 seconds<br/><br/>The edge worker controls the cache key,<br/>enabling personalised caching"]
    end

This diagram shows three edge state options. Key-value stores like Cloudflare Workers KV provide eventual consistency with sub-10ms reads but 10-60 second write propagation - good for config and feature flags but not counters. Durable Objects provide strong consistency through single-threaded actors - good for rate limiting and counters but the object lives in one location so distant users get higher latency. Edge cache provides sub-millisecond reads on cache hits with TTL-based staleness - the edge worker controls the cache key to enable personalised caching.

Warning: Eventually consistent KV at the edge can cause subtle bugs. Imagine you deploy a feature flag to Workers KV that disables a broken feature. The write propagates in 30-60 seconds. During that window, some edge locations still serve the old flag value and users there see the broken feature. For safety-critical flags (kill switches, rate limits), use a synchronous mechanism - propagate through the request path (header from origin) rather than relying on KV propagation.

Edge Architecture Patterns

Pattern 1: Edge Gateway

graph LR
    USER2["User"] --> EDGE2["Edge Worker<br/>(auth, routing, rate limiting)"]
    EDGE2 -->|"Authenticated,<br/>rate-checked request"| ORIGIN2["Origin API<br/>(business logic, database)"]
    ORIGIN2 --> EDGE2
    EDGE2 --> USER2

This diagram shows the edge gateway pattern. The user's request hits the edge worker, which handles authentication, routing, and rate limiting. The authenticated and validated request is forwarded to the origin API for business logic and database access. The response flows back through the edge.

The edge handles cross-cutting concerns - auth, rate limiting, routing, header manipulation - and forwards valid requests to origin. This is the most common pattern and the easiest to adopt incrementally.

Pattern 2: Edge Compute with Origin Fallback

graph TD
    USER3["User request"] --> EDGE3["Edge Worker"]
    EDGE3 --> CHECK{"Can edge<br/>serve this?"}
    
    CHECK -->|"Yes (cache hit or<br/>local computation)"| SERVE["Edge serves response<br/>directly - 5ms latency"]
    
    CHECK -->|"No (cache miss,<br/>needs fresh data)"| FETCH["Fetch from origin<br/>Cache response at edge<br/>for next user<br/>— 160ms latency"]
    
    FETCH --> SERVE2["Serve response<br/>+ populate edge cache"]

This diagram shows edge compute with origin fallback. The edge worker checks whether it can serve the request from cache or local computation. If yes, it serves directly with 5 millisecond latency. If not, it fetches from origin, caches the response at the edge for the next user, and serves the response with 160 millisecond latency.

Pattern 3: Full Edge Application

graph TD
    USER4["User"] --> EDGE4["Edge Worker<br/>(full application logic)"]
    EDGE4 --> EDGE_DB["Edge Database<br/>(D1, Turso, Neon Edge)"]
    EDGE4 --> EDGE_KV["Edge KV Store"]
    EDGE4 --> EDGE_QUEUE["Edge Queue<br/>(async tasks)"]
    
    EDGE_DB -->|"Async replication"| CENTRAL_DB["Central Database<br/>(source of truth)"]
    
    EDGE4 --> USER4

This diagram shows the full edge application pattern. The edge worker runs complete application logic and accesses an edge database, edge KV store, and edge queue - all at the edge. The edge database replicates asynchronously to a central database that serves as the source of truth. The user interacts entirely with the edge, with no synchronous round-trip to origin.

This is the most aggressive pattern - running your entire application at the edge with edge-native databases. Emerging platforms like Cloudflare's D1 (SQLite at the edge) and Turso (distributed libSQL) make this possible for certain workloads. The trade-off: eventual consistency between edge database replicas and the central source of truth.

Interview: "Where would you put compute in your architecture - origin or edge?" This question tests whether you think about latency budgets, not just architecture patterns. The strong answer: "I'd evaluate each operation against three criteria. First, does it need fresh data from a central database? If yes, it stays at origin. Second, is it latency-sensitive for the end user? Authentication, A/B test assignment, and geo-routing are excellent edge candidates because they're on the critical path of every request and can be computed with only the request context - no database round-trip. Third, is the compute lightweight enough for edge constraints - typically 50ms CPU limit and no persistent connections? Heavy operations like report generation or ML inference stay at origin. For our design, I'd put JWT validation, bot detection, and static content routing at the edge, keeping business logic and database writes at origin."

The Production War Story

At Amazon, we migrated the authentication layer for a high-traffic consumer API from origin-based validation to edge-based validation using CloudFront Functions. The auth check was simple: validate a JWT signature, check expiration, extract the user ID, and forward to origin with a trusted header.

The migration cut median API latency by 120ms for international users - the JWT validation no longer required a round-trip to us-east-1. Great outcome. Then we hit the problem.

JWT signing key rotation. Every 90 days, our auth service rotated the RSA key pair and published the new public key. The old key remained valid for 24 hours after rotation for graceful transition. CloudFront Functions had no network access - they couldn't call the JWKS endpoint to discover the new key. We'd baked the public key into the function code and redeployed on rotation day.

The rotation happened at 2 AM Pacific. The deployment took 15 minutes to propagate to all CloudFront edge locations. During those 15 minutes, some edge locations had the old key, some had the new key, and users whose JWTs were signed with the new key got 401 errors at edge locations still running the old function version.

The fix: embed both the current and next signing key in the function code, deploy the next key a week before rotation, and try both keys during validation. This way, during the 15-minute deployment window, every edge location has both keys. After rotation, the old "next" key becomes "current" and a new "next" key is pre-deployed.

The lesson: edge compute removes your origin from the hot path - which is the point - but it also removes your origin's ability to dynamically respond to config changes. Anything that changes at the origin (keys, flags, config) must be pushed to the edge proactively, and the propagation delay creates a window of inconsistency you must design around.

Six Takeaways

1. The edge is a proximity-capability trade-off. Edge POPs are close to users (5-20ms) but have limited compute and no primary database. Origin is far (50-200ms) but has full capability. The architectural decision is about which work happens where.

2. Cold start separates the platforms. Cloudflare Workers (~0ms via V8 isolates) and Fastly Compute (~0ms via Wasm) are fundamentally different from Lambda@Edge (1-3 seconds) for latency-sensitive use cases.

3. Auth, routing, rate limiting, and caching are natural edge workloads. They're cross-cutting, latency-sensitive, lightweight, and don't need the primary database. Move them to the edge first.

4. Edge state is the hard problem. KV stores are fast but eventually consistent. Durable Objects provide strong consistency but pin state to one location. Edge cache is fast but TTL-based. Choose based on your consistency requirements.

5. Edge compute removes the origin from dynamic config updates. Signing keys, feature flags, and config changes must be proactively pushed to the edge. Propagation delay creates inconsistency windows you must design around.

6. Start with edge gateway, evolve toward full edge. The edge gateway pattern (auth + routing at edge, business logic at origin) gives the biggest latency win for the smallest migration effort. Full edge applications are emerging but require edge-native databases and eventual consistency tolerance.

Exercises

Exercise: Starter: Your web application serves users across North America, Europe, and Asia from a single data centre in us-east-1. (a) Estimate the round-trip latency for users in London, Mumbai, and Tokyo. (b) A page load makes 4 sequential API calls. What's the total network latency for each region? (c) You move JWT validation to a Cloudflare Worker. How much latency does this save per API call for each region? (d) Which of these workloads should you move to the edge: user login, product search, order placement, image serving, A/B test assignment? Justify each.

Exercise: Intermediate: Design an edge architecture for a content platform (like Medium) that serves 10 million daily readers across 50 countries. (a) Article pages are read-heavy and change infrequently - what caching strategy do you use at the edge? What TTL? How do you invalidate when the author edits? (b) The platform has a paywall - some articles are free, some require a subscription. The paywall check needs the user's subscription status. Design the edge worker flow: how do you check subscription status without a round-trip to origin on every request? (c) The platform does A/B testing on article layouts. Design the edge-based A/B assignment: how do you ensure a user always sees the same variant? (d) The platform wants to add real-time view counts ("1,234 people reading this now"). Can this run at the edge? What edge state primitive would you use?

Exercise: Advanced: You're building a multiplayer game backend that needs sub-50ms latency globally. (a) Player matchmaking needs to group players by region. Design the edge-based matchmaking: where does the matchmaking logic run, and where is the match state stored? (b) In-game state updates (player positions, actions) happen 30 times per second per player. Can this traffic flow through edge workers, or do you need dedicated regional game servers? Calculate the compute cost at Cloudflare Workers pricing for 100,000 concurrent players. (c) The game has an inventory system - players trade items. This requires strong consistency (no item duplication). Design the architecture: which operations run at the edge, which at origin, and how do you handle a player in Tokyo trading with a player in London? (d) Anti-cheat detection needs to analyse player input patterns in real-time. Where does this run, and how do you balance the latency requirement (real-time) with the compute requirement (ML model inference)?

Back to System Design Advanced