Free lesson · System Design Interviews

Step 8: Bottlenecks and Trade-offs

Hard10 min readsystem-designinterview-prep

This is the final step, and in many interviews it's the most important one. The interviewer has watched you build a system for 35 minutes. Now they want to see if you can step back and think critically about your own design. Where are the weak spots? What did you trade away? What would break first under unexpected load? What would you do differently with more time?

This is the section that separates candidates who can build systems from candidates who can own systems in production. Let me walk through the bottlenecks and trade-offs in our URL shortener design.

Bottleneck Analysis

Bottleneck 1: Single Point of Write - The Database Primary

Every URL creation goes through a single PostgreSQL primary. Even though 1-3K QPS is within capacity, this is still a single point of failure and a scaling ceiling.

flowchart TD
    W1["Write Server 1"] --> PRIMARY[("PostgreSQL Primary<br/>Single point of write")]
    W2["Write Server 2"] --> PRIMARY
    W3["Write Server 3"] --> PRIMARY
    
    PRIMARY -->|"Replication"| R1[("Replica 1")]
    PRIMARY -->|"Replication"| R2[("Replica 2")]
    
    RISK["If Primary fails:<br/>- Writes stop completely<br/>- Failover takes 10-30 seconds<br/>- Possible data loss (async replication)"]

Bottleneck 1: the database primary. All URL creation writes go through a single PostgreSQL primary. If the primary fails, all writes stop for 10 to 30 seconds during failover. With asynchronous replication, writes that haven't replicated yet may be lost. Mitigation options include synchronous replication to prevent data loss (at the cost of higher write latency), automated failover with tools like Patroni, or database sharding to eliminate the single primary.

Impact: If the primary fails, no new URLs can be created for 10-30 seconds during failover. Reads continue from replicas and cache.

Mitigation options:

My recommendation in an interview: "I'd use Patroni for automated failover with synchronous replication to one replica. This gives us zero data loss and 10-15 second failover. I wouldn't shard yet - the write throughput doesn't require it. Sharding is the plan for 10x scale."

Bottleneck 2: Hot Keys in Redis

A single viral URL can receive 50,000+ reads per second. In a sharded Redis cluster, all requests for one key hit the same shard, creating a hot shard.

Impact: The hot shard's latency degrades from <1ms to 5-10ms. Other keys on the same shard also experience degraded latency. If the shard saturates, cache misses increase for all keys on that shard.

Mitigation: Local in-memory cache on each read server for the top 1,000 URLs. Viral URLs are served from local memory, never reaching Redis.

Bottleneck 3: Counter Range Exhaustion

If a write server crashes, the unused portion of its counter range is lost. If the range allocation service is unavailable, write servers exhaust their current range and can't create new URLs.

Impact: At 1,000 QPS, a 1-million-code range lasts ~16 minutes. If the range allocation service is down for more than 16 minutes, URL creation stops.

Mitigation:

• Allocate larger ranges (10 million) to provide 2.7 hours of buffer
• Run the range allocation service with redundancy (2-3 instances)
• Wasted codes from crashed servers are negligible: even losing 10 million codes consumes 0.0003% of the 3.5 trillion code space

Bottleneck 4: Analytics Pipeline Lag

Click events flow through Kafka asynchronously. During traffic spikes, the analytics pipeline can lag - the click count shown to users may be minutes behind reality.

Impact: A user creates a short URL, shares it, and checks the stats page. They see 0 clicks even though hundreds of people have already clicked. This is a user experience issue, not a system correctness issue.

Mitigation:

• Accept eventual consistency for analytics (most systems do)
• For real-time approximate counts, use a Redis counter that's incremented synchronously on the redirect path, separate from the durable analytics pipeline
• Display "~1,200 clicks" with a tilde to signal the count is approximate

Trade-off Analysis

Every design decision trades something for something else. Here are the key trade-offs in our design and what we chose.

Trade-off 1: 302 vs 301 Redirects

flowchart LR
    subgraph CHOSE["What We Chose: 302"]
        A1["✓ Every click tracked"]
        A2["✓ Can change destination URL"]
        A3["✗ Higher server load"]
        A4["✗ Slightly slower (server round-trip)"]
    end
    
    subgraph ALTERNATIVE["Alternative: 301"]
        B1["✗ Only first click tracked"]
        B2["✗ Can't change destination (cached)"]
        B3["✓ Lower server load"]
        B4["✓ Faster repeat visits"]
    end

Trade-off 1: we chose 302 temporary redirects over 301 permanent redirects. The benefit is that every click is tracked and we can change the destination URL. The cost is higher server load because browsers don't cache the redirect. If analytics weren't a requirement, 301 would reduce server load significantly because returning users would be redirected by their browser cache without hitting our servers at all.

What we gave up: With 301, the browser caches the redirect. Repeat visitors never hit our servers - they're redirected locally by their browser. This could reduce our read QPS by 50-70%. We chose analytics accuracy over infrastructure cost savings.

Trade-off 2: Write-Through Caching vs Write-Behind

What we gave up: Write-behind would reduce creation latency by ~5ms (the cache write is faster than the DB write). But we'd risk losing URL mappings if Redis crashes before the async write reaches PostgreSQL. For a URL shortener, data durability is more important than 5ms of write latency.

Trade-off 3: Counter-Based vs Hash-Based Code Generation

What we gave up: Hash-based codes look random, which prevents enumeration attacks (where someone scans sequential codes to discover all URLs). Counter-based codes are sequential and guessable. We chose performance (no collision checks) over security. To mitigate, we'd apply a bijective shuffle to the counter value before Base62 conversion.

Trade-off 4: Async Analytics vs Synchronous Analytics

What we gave up: Real-time accurate click counts. A user checking their stats sees data that's seconds to minutes behind. We chose redirect latency over analytics freshness - the right trade-off because the redirect path is 1000x more frequent than the stats path.

Trade-off 5: Single Region vs Multi-Region

What we deferred: Multi-region is the 100x scaling answer, not the day-one architecture. Starting single-region keeps the system simple and correct. CDN handles the global latency problem for the most common operation (redirects) without the complexity of multi-region databases.

What Would Break First?

If I had to predict the first production incident for this system, here's my ordered list:

1. A viral URL causes a hot key in Redis. One URL gets 100K clicks/sec, saturating a single Redis shard. Other URLs on the same shard experience elevated latency. Users report slow redirects for unrelated URLs.

2. The analytics pipeline falls behind during sustained high traffic. Kafka consumer lag grows to minutes. Users see stale click counts. Not a system failure, but a user experience issue that generates support tickets.

3. Database connection pool exhaustion during a cache failure. Redis failover takes 15 seconds. During those 15 seconds, all 100K redirects/sec hit the database directly. The database connection pool (typically 100-200 connections) is overwhelmed. Redirects start timing out.

4. Short code collision in hash-based generation (if we used that approach). Two URLs get the same 7-character hash truncation. The second insert fails with a unique constraint violation. The retry logic handles it, but at high write rates, collision probability increases and retry storms can cascade.

5. Custom code squatting. A user registers short.ly/google, short.ly/amazon, short.ly/facebook - claiming valuable short codes for spam or resale. Not a technical failure, but a business problem that requires a reserved-words list and potentially a trademark policy.

Interview: How to present this section: "Let me walk through what I think would break first in production. The most likely incident is a viral URL creating a hot key in Redis - one URL getting 100K clicks per second, all hitting the same shard. I'd mitigate this with local caching on each read server. The second risk is analytics lag during traffic spikes - users see stale click counts. The third is database connection exhaustion during a Redis failover - all traffic suddenly hitting the database. I'd mitigate with connection pooling limits and circuit breakers."

If I Had More Time - Improvements

These are the enhancements I'd propose if the interviewer gives you the last 2-3 minutes for open discussion:

1. Link Preview and Metadata

When sharing a short URL on Slack or Twitter, the platform fetches an Open Graph preview (title, description, image). We could cache OG metadata alongside the URL mapping and return it via a preview API endpoint. This makes short URLs more shareable and trustworthy.

2. Abuse Detection

Short URLs are frequently used for phishing. A system that scans destination URLs against known phishing databases (Google Safe Browsing API) before creating the short URL protects users and the platform's reputation.

3. A/B Testing with Short URLs

A single short code could redirect to different long URLs based on a percentage split. short.ly/campaign sends 50% of traffic to page A and 50% to page B. This turns the URL shortener into a lightweight experimentation platform.

4. Detailed Analytics

Expand from click count to click breakdown: by country (from IP geolocation), by device (from User-Agent parsing), by referrer, by time of day. Store click events in a columnar database (ClickHouse or BigQuery) optimized for analytical queries.

The Complete Design Summary

Let me bring together everything from all 8 steps:

flowchart TB
    subgraph GLOBAL["Global Layer"]
        DNS["GeoDNS"]
        CDN["CDN Edge Nodes"]
    end
    
    subgraph ORIGIN["Origin (US-East)"]
        LB["Application<br/>Load Balancer"]
        
        subgraph WRITE["Write Path"]
            WS["Write Servers<br/>(3 instances)"]
            CODEGEN["Code Generator<br/>(counter ranges)"]
        end
        
        subgraph READ["Read Path"]
            RS["Read Servers<br/>(10+ instances)"]
        end
        
        CACHE[("Redis Cluster<br/>3 shards<br/>6 nodes")]
        DB[("PostgreSQL<br/>Primary + 2 Replicas")]
        
        subgraph ASYNC["Async Pipeline"]
            KAFKA["Kafka"]
            WORKERS["Analytics<br/>Workers"]
        end
    end
    
    DNS --> CDN
    CDN -->|"Cache MISS"| LB
    LB --> WS
    LB --> RS
    RS --> CACHE
    CACHE -.->|"Cache MISS"| DB
    WS --> CODEGEN
    WS --> DB
    WS --> CACHE
    RS -->|"Click events"| KAFKA
    KAFKA --> WORKERS
    WORKERS --> DB

Complete URL shortener architecture. At the global layer, GeoDNS routes users to CDN edge nodes. Cache hits are served from the edge in under 5 milliseconds. Cache misses reach the origin in US-East through an Application Load Balancer. The write path has 3 write servers with a code generator using pre-allocated counter ranges, writing to PostgreSQL and Redis. The read path has 10 or more read servers checking the Redis Cluster first - a 3-shard, 6-node cluster. Cache misses query PostgreSQL. Click events flow asynchronously through Kafka to analytics workers that batch-update click counts in the database. The full system handles 100,000 redirect QPS at under 50 milliseconds p99 latency.

Key Takeaways

1. Know your bottlenecks before the interviewer asks. The three biggest risks are hot keys in Redis, analytics pipeline lag, and database connection exhaustion during cache failover. Name them proactively.

2. Every design decision is a trade-off - articulate both sides. 302 vs 301, write-through vs write-behind, counter vs hash, async vs sync analytics. The interviewer doesn't care which you chose - they care that you know what you traded away.

3. Start simple, scale in stages. Single region → CDN → multi-region. Single database → read replicas → sharding. Single Redis → Redis Cluster. Each step is triggered by a specific bottleneck, not by speculation.

4. Async analytics is the right default for read-heavy systems. Don't put analytics on the critical redirect path. Accept eventual consistency for click counts in exchange for <10ms redirect latency.

5. The "what would break first" question is gold. Interviewers love candidates who can predict failure modes. Practice identifying the weakest link in your design - it shows production experience.

6. End with improvements, not apologies. "If I had more time, I'd add abuse detection and detailed analytics" is stronger than "I know this design isn't perfect." Show that you see the next iteration, not that you're insecure about the current one.

Exercise: Exercise 1: A user reports that short.ly/important-link redirects to a phishing page. Design the incident response: How do you disable the URL immediately? How do you prevent the cached redirect from continuing to serve the phishing URL across CDN and Redis? What's the maximum time between report and mitigation? (Hint: cache invalidation across CDN and Redis must happen simultaneously.)

Exercise 2: The business team wants to add paid tiers - free users get URLs that expire in 30 days, paid users get URLs that never expire. How does this change the database schema, the cache TTL strategy, and the storage growth estimation? Does the cleanup job need to be tier-aware?

Exercise 3: Write a post-mortem for the following incident: "At 2:00 AM UTC, the Redis primary in Shard 2 ran out of memory. Shard 2 stopped accepting writes. All URLs with short codes hashing to Shard 2 returned stale data from read replicas for 45 minutes until an on-call engineer manually evicted cold keys. Root cause: a batch job loaded 50 million expired URL mappings into cache without TTL." Include: timeline, impact, root cause, contributing factors, corrective actions.

Back to System Design Interviews