Free lesson · System Design Interviews

Step 7: Deep Dive - Scaling Design

Hard13 min readsystem-designinterview-prep

Here's something counterintuitive about scaling Pastebin: the QPS numbers are low enough that a single well-configured server could handle the entire service. Fifty writes per second, 250 reads per second - that's laptop-scale traffic. So why does this system need a scaling discussion at all?

Because Pastebin's scaling challenge isn't about request throughput - it's about content delivery, storage growth, and the operational complexity that comes from separating content storage from metadata storage. When an interviewer asks "how does this scale?", they're not asking how you handle 10 million QPS. They're asking how you handle 10x more storage, 10x more bandwidth, global users, and the operational challenges of running a system where the heaviest component (S3 content) scales independently from the lightest component (PostgreSQL metadata).

At Amazon, I've seen teams over-engineer systems for QPS that never materialised while under-engineering for storage growth that bankrupted their service budget. Pastebin is a storage-and-bandwidth scaling problem, and I want to make sure you approach it that way.

What Actually Needs to Scale?

Let me rank the scaling pressures from most urgent to least:

flowchart TB
    subgraph CRITICAL["Primary Scaling Concerns"]
        BW["1. Bandwidth<br/>Content delivery at global scale<br/>Viral paste spikes"]
        STORAGE["2. Storage Growth<br/>19 TB over 5 years<br/>Cost optimization"]
    end
    
    subgraph MODERATE["Secondary Scaling Concerns"]
        DB["3. Database Growth<br/>Metadata table: 500M+ rows<br/>Query performance"]
        CLEANUP["4. Cleanup Operations<br/>Millions of expired pastes<br/>S3 deletion throughput"]
    end
    
    subgraph LOW["Low Concern"]
        QPS["5. Request Throughput<br/>250 read QPS<br/>Single server handles this"]
    end

Scaling priority ranking for Pastebin. The primary concerns are bandwidth - delivering content globally and handling viral paste spikes - and storage growth - managing 19 terabytes over 5 years cost-effectively. Secondary concerns are database growth as the metadata table exceeds 500 million rows and cleanup operations that need to delete millions of expired pastes from S3. Request throughput is the lowest concern - 250 read QPS is well within a single server's capacity. This is the opposite of URL Shortener, where QPS was the primary scaling challenge.

Interview: Set the framing early: "Pastebin scales along a different axis than URL Shortener. URL Shortener's challenge was handling 100K QPS with tiny payloads. Pastebin's challenge is delivering variable-size content globally, managing storage growth, and optimising costs. Let me walk through each concern in order of urgency."

Scaling Content Delivery (Bandwidth)

The Bandwidth Growth Model

At 10x our baseline traffic:

10x baseline:
- Write QPS: 500 avg / 1,500 peak
- Read QPS: 2,500 avg / 7,500 peak
- Read bandwidth: 25 MB/sec avg / 75 MB/sec peak
- Daily content ingestion: 500 GB/day

Viral spike on top:
- A 1 MB paste gets 1,000 views/sec
- Spike bandwidth: 1 GB/sec from one paste
- Origin servers can't handle this - CDN must absorb it

CDN Scaling Strategy

flowchart TB
    subgraph EDGE_NODES["CDN Edge Layer (200+ PoPs globally)"]
        TOKYO["Tokyo PoP<br/>Local cache"]
        LONDON["London PoP<br/>Local cache"]
        NYC["NYC PoP<br/>Local cache"]
        SAO["São Paulo PoP<br/>Local cache"]
    end
    
    subgraph REGIONAL["Regional Edge Cache (3 regions)"]
        REC_US["US Regional Cache<br/>Aggregates edge misses"]
        REC_EU["EU Regional Cache<br/>Aggregates edge misses"]
        REC_AP["AP Regional Cache<br/>Aggregates edge misses"]
    end
    
    subgraph ORIGIN["Origin (1 region)"]
        ORIGIN_API["API Servers<br/>+ Redis + S3"]
    end
    
    TOKYO --> REC_AP
    LONDON --> REC_EU
    NYC --> REC_US
    SAO --> REC_US
    
    REC_US --> ORIGIN
    REC_EU --> ORIGIN
    REC_AP --> ORIGIN

CDN scaling with regional edge caches. CloudFront has over 200 Points of Presence globally. When a user in Tokyo requests a paste, the Tokyo edge node serves it from local cache if available. If not, the request goes to a regional edge cache in Asia Pacific, which aggregates misses from multiple edge nodes in the region. Only if the regional cache also misses does the request reach the origin servers in the US. This two-tier CDN architecture reduces origin traffic by 90 to 95 percent for popular content, because even if a specific edge node doesn't have the content, the regional cache likely does.

CloudFront's regional edge caches are automatic - you don't configure them. They're intermediate caches between edge PoPs and your origin that aggregate cache misses from multiple edge nodes. This means:

• Popular paste viewed globally: Each of the 200+ edge nodes fetches the content once from the regional cache. The regional cache fetches once from origin. Total origin requests: ~3 (one per region), not ~200.
• Unpopular paste viewed once from Tokyo: Edge miss → regional miss → origin fetch. Three hops, but this only happens once for this paste in this region.

CDN-origin traffic after scaling:

Without CDN: 7,500 reads/sec × 10 KB = 75 MB/sec at origin
With CDN (95% hit rate): 375 reads/sec × 10 KB = 3.75 MB/sec at origin

CDN absorbs: 71.25 MB/sec of bandwidth
Origin sees: 3.75 MB/sec - a single server handles this

Origin Shield for Cost Optimisation

At 10x scale, even CDN origin fetches add up. CloudFront Origin Shield adds a centralised cache between the regional edge caches and your origin, reducing origin fetches further:

Without Origin Shield:
3 regional caches × 1 fetch per unique paste = 3 origin fetches per paste

With Origin Shield:
3 regional caches → 1 Origin Shield → 1 origin fetch per paste

Savings: 67% fewer origin S3 GETs
At 10x: saves ~$600/month in S3 GET costs

Tip: Mention Origin Shield in the interview. It's a specific, tactical optimisation that shows you understand CDN architecture beyond "just put CloudFront in front of S3." Most candidates know CDN caching but don't know about the regional-cache-to-origin pipeline or how to optimise it.

Scaling Storage

S3 Storage Growth Management

S3 handles storage scaling natively - you never run out of space and there's no provisioning. But cost optimisation matters at scale.

Year 1: ~8 TB active storage → ~
84/month
Year 3: ~14 TB active storage → ~$322/month
Year 5: ~19 TB active storage → ~$437/month

With S3 Intelligent-Tiering:
Automatically moves objects between access tiers
Estimated savings: 30-40% for older content

Year 5 with Intelligent-Tiering: ~$280/month (vs $437)

flowchart TD
    subgraph TIERS["S3 Intelligent-Tiering"]
        FREQ["Frequent Access<br/>First 30 days<br/>$0.023/GB"]
        INFREQ["Infrequent Access<br/>30-90 days<br/>$0.0125/GB"]
        ARCHIVE["Archive Instant Access<br/>90+ days<br/>$0.004/GB"]
    end
    
    FREQ -->|"Auto-transition<br/>after 30 days<br/>no access"| INFREQ
    INFREQ -->|"Auto-transition<br/>after 90 days<br/>no access"| ARCHIVE
    ARCHIVE -->|"Access detected"| FREQ
    
    COST["Year 5 savings: ~
57/month<br/>compared to all Standard"]

S3 Intelligent-Tiering for storage cost optimisation. New paste objects start in the Frequent Access tier at $0.023 per gigabyte per month. After 30 days without access, objects automatically move to Infrequent Access at about half the cost. After 90 days without access, they move to Archive Instant Access at about one-sixth the cost. If an object is accessed again, it automatically moves back to Frequent Access. At year 5, this saves about 
57 per month compared to keeping everything in Standard - meaningful at scale but not a make-or-break cost.

Content Deduplication at Scale

At 5 million pastes per day, deduplication savings grow:

Conservative dedup rate: 10% of pastes are duplicates
Savings: 50 GB/day × 10% = 5 GB/day = 1.8 TB/year
At S3 pricing: ~$42/year saved

Aggressive dedup rate: 25% (realistic for common code snippets)
Savings: 50 GB/day × 25% = 12.5 GB/day = 4.5 TB/year
At S3 pricing: ~
04/year saved

At these numbers, deduplication saves $42-104/year in storage - not worth the implementation complexity at our scale. But at 100x scale (50 TB/year), deduplication saves $4,200-10,400/year, making it worthwhile.

Interview: Frame it as "when, not if": "Content deduplication saves ~10% of storage. At our current scale of 8 TB, that's

8/month - not worth the complexity. But I've added the content hash to the schema from day one so we can enable dedup later without a data migration. At 10x scale, the savings justify the implementation."

Scaling the Database

PostgreSQL Growth at Scale

Records per day: 5 million
Records per year: 1.825 billion
Active records (after cleanup): ~150 million (year 1) → ~500 million (year 5)
Metadata size: 500M × 300 bytes = ~150 GB

150 GB is well within a single PostgreSQL instance's capacity, but query performance degrades as the table grows. The specific queries that suffer:

Table Partitioning

At 500 million rows, partitioning by created_at improves cleanup query performance dramatically:

-- Range partition by month
CREATE TABLE paste_metadata (
    id              BIGSERIAL,
    paste_id        VARCHAR(8) NOT NULL,
    created_at      TIMESTAMP NOT NULL DEFAULT NOW(),
    expires_at      TIMESTAMP,
    deleted_at      TIMESTAMP,
    -- ... other columns ...
    PRIMARY KEY (id, created_at)
) PARTITION BY RANGE (created_at);

-- Create monthly partitions
CREATE TABLE paste_metadata_2026_01 PARTITION OF paste_metadata
    FOR VALUES FROM ('2026-01-01') TO ('2026-02-01');
CREATE TABLE paste_metadata_2026_02 PARTITION OF paste_metadata
    FOR VALUES FROM ('2026-02-01') TO ('2026-03-01');
-- ... one per month, automated via pg_partman

-- Benefits:
-- 1. Cleanup queries scan only relevant partitions
-- 2. Old partitions can be dropped entirely (fastest deletion)
-- 3. Each partition is ~15M rows (much faster queries)
-- 4. Vacuum operates on smaller tables

flowchart TB
    subgraph UNPARTITIONED["Without Partitioning"]
        BIG_TABLE["paste_metadata<br/>500M rows<br/>150 GB<br/>Slow cleanup queries"]
    end
    
    subgraph PARTITIONED["With Monthly Partitioning"]
        P1["2026_01<br/>15M rows"]
        P2["2026_02<br/>15M rows"]
        P3["2026_03<br/>15M rows"]
        P_DOTS["..."]
        P12["2026_12<br/>15M rows"]
    end
    
    BENEFIT["Cleanup: DROP old partitions<br/>instantly (vs DELETE millions of rows)<br/><br/>Queries: scan only relevant<br/>month's partition"]

Database table partitioning for scale. Without partitioning, the paste_metadata table grows to 500 million rows at year 5 - 150 gigabytes. Cleanup queries that scan for expired records slow down as the table grows. With monthly partitioning, each partition holds about 15 million rows. Cleanup can drop entire old partitions instantly instead of deleting millions of rows one by one. Queries that filter by creation date automatically scan only the relevant partition. This is the most impactful database scaling technique for a time-series workload like Pastebin.

Tip: Partitioning is the database scaling answer for Pastebin, not sharding. The metadata fits on a single instance. The problem is that cleanup operations slow down as the table grows. Partitioning solves this by making cleanup an O(1) partition drop instead of an O(n) row delete. Mention partitioning before sharding - it's the right tool for this problem.

Read Replicas for Query Isolation

At 10x scale, add a read replica to separate the read path from the cleanup path:

Primary: Handles writes (500 QPS) and cleanup (DELETE/UPDATE)
Replica 1: Handles reads (2,500 QPS) - cache misses only (~250 QPS after Redis)

The cleanup worker's bulk DELETE operations can briefly lock rows on the primary. With a read replica, the read path is insulated from cleanup lock contention.

Scaling the Write Path

Upload Processing at Scale

At 10x (500 writes/sec, 5 MB/sec upload bandwidth):

Write path operations per paste:
1. Receive content (10 KB avg) → ~0.5ms
2. Gzip compress → ~1ms for 10 KB
3. Upload to S3 → ~20-50ms
4. Insert metadata to PostgreSQL → ~2-5ms
5. Populate Redis cache → ~1ms
Total: ~25-60ms per paste

At 500 QPS: 500 × 60ms = 30 seconds of concurrent processing
With 10ms of actual CPU per request: need 5 CPU cores

A single 8-core server handles 500 write QPS with headroom. At 10x, two write servers provide redundancy without excess cost.

S3 Write Throughput

S3 supports at least 3,500 PUT requests per second per prefix. At 500 writes/sec with a flat key structure (pastes/{paste_id}), we're well within limits. At 100x (5,000 writes/sec), we'd approach the per-prefix limit. Solutions:

Option 1: Add prefix diversity
pastes/{first-2-chars-of-id}/{paste_id}
→ 62^2 = 3,844 prefixes → 3,844 × 3,500 = 13.4M PUTs/sec capacity

Option 2: Multiple S3 buckets
paste-content-1, paste-content-2, paste-content-3
Route by hash of paste_id

Both are trivial at Pastebin's scale - this won't be a bottleneck until extreme scale.

Multi-Region Scaling

At 100x scale, you need to serve users globally with low latency. The CDN handles content delivery, but API responses (metadata) still come from a single origin region.

Architecture for Global Users

flowchart TB
    subgraph GLOBAL["Global Layer"]
        DNS["GeoDNS"]
        CDN["CDN (200+ PoPs)<br/>Handles raw content globally"]
    end
    
    subgraph US["US-East (Primary)"]
        API_US["API Servers"]
        REDIS_US[("Redis")]
        DB_US[("PostgreSQL Primary")]
        S3_US[("S3 Primary Bucket")]
    end
    
    subgraph EU["EU-West (Read Replica)"]
        API_EU["Read API Servers"]
        REDIS_EU[("Redis")]
        DB_EU[("PostgreSQL Replica")]
        S3_EU[("S3 Replicated")]
    end
    
    DNS -->|"Content: CDN"| CDN
    DNS -->|"Write API: US-East"| API_US
    DNS -->|"Read API: nearest"| API_EU
    
    DB_US -->|"Async replication"| DB_EU
    S3_US -->|"Cross-region replication"| S3_EU

Multi-region architecture for Pastebin at 100x scale. The CDN handles raw content delivery globally from 200-plus edge nodes - this covers the most bandwidth-intensive operation. For API requests, GeoDNS routes write operations to US-East where the primary database lives, and read operations to the nearest regional API cluster. EU-West has read-only API servers with a PostgreSQL read replica and a Redis cache. S3 Cross-Region Replication ensures content is available in both regions. This architecture optimises for the common case - reading pastes - while accepting slightly higher latency for the rare case - creating pastes.

Why writes stay in one region: At 500 writes/sec (10x), the write throughput is low enough for a single primary. Multi-primary adds complexity (conflict resolution, distributed transactions) that's not justified until 50,000+ writes/sec.

S3 Cross-Region Replication (CRR): Replicates S3 objects from the primary bucket to regional buckets. New pastes are available in the secondary region within 15-30 seconds. Regional API servers read from their local S3 bucket, avoiding cross-region S3 latency.

The Real Multi-Region Decision

For Pastebin specifically, multi-region API servers may not be necessary. Here's why:

CDN handles 95% of content reads globally (raw endpoint)
API metadata response is ~300 bytes (tiny, fast even cross-region)
Write latency is user-initiated (not latency-critical)

True multi-region API adds:
- 2x infrastructure cost
- Cross-region database replication lag
- Deployment complexity

For 2x cost, you get ~50ms improvement on metadata reads

In many cases, CDN-only global scaling is sufficient for Pastebin. The API stays single-region, but the CDN serves content globally. This is the pragmatic answer that shows cost-awareness.

Interview: What to say: "For Pastebin, I'd scale globally with CDN first and defer multi-region API deployment. The CDN handles 95% of content reads from edge nodes. The remaining API requests are for small metadata payloads - 300 bytes served cross-region in ~150ms. Multi-region API servers would improve metadata latency by ~50ms at 2x infrastructure cost. I'd only invest in multi-region if the business justifies it - for example, if we need data residency compliance in the EU."

Scaling the Cleanup System

At 5 million pastes/day with 60% expiring within 30 days, the cleanup worker processes:

Expired pastes per day: 5M × 0.6 = 3M pastes
S3 DELETE operations: 3M/day → ~35/sec

S3 DELETE throughput: 3,500 DELETEs/sec per prefix
35/sec is trivially within limits.

At 10x: 350/sec - still trivially within limits.
At 100x: 3,500/sec - approaching per-prefix limit.

Batch S3 Deletion for Efficiency

S3's DeleteObjects API allows deleting up to 1,000 objects per request:

def batch_delete_s3_objects(paste_ids):
    """Delete expired paste content in batches of 1,000."""
    for batch in chunks(paste_ids, 1000):
        s3.delete_objects(
            Bucket="paste-content",
            Delete={
                'Objects': [{'Key': f'pastes/{pid}'} for pid in batch],
                'Quiet': True  # Don't return individual success/failure
            }
        )

At 3 million deletions per day in batches of 1,000, that's 3,000 API calls per day - roughly one every 30 seconds. Trivial.

S3 Lifecycle Rules as a Complementary Cleanup

S3 lifecycle rules can automatically delete objects after a specified duration. For pastes with fixed expiration:

Lifecycle rule: Delete objects with tag "expiration=30d" after 30 days

This complements the cleanup worker - even if the worker falls behind, the lifecycle rule ensures objects are eventually deleted. Belt-and-suspenders for storage cost control.

The Scaling Roadmap

Warning: Don't jump to the 100x architecture. Start with 1x. When the interviewer asks about scale, walk through 10x - CDN optimisation, Redis Cluster, partitioned database. Only sketch 100x if they push further. Jumping to multi-region on the first answer signals that you over-engineer.

Key Takeaways

1. Pastebin scales on bandwidth and storage, not QPS. 250 read QPS is trivial. 75 MB/sec peak bandwidth and 19 TB storage over 5 years are the real scaling challenges. Frame every scaling decision around content delivery and storage cost.

2. CDN is the primary scaling mechanism. CloudFront with Origin Shield absorbs 95% of content reads at edge nodes. This single component handles global scaling, viral spikes, and bandwidth offloading - more impact than any other scaling investment.

3. Table partitioning before database sharding. The metadata table grows to 500M rows, but fits on one instance. Partitioning by month makes cleanup instant (drop partitions) and queries faster (scan smaller tables). Sharding is unnecessary at Pastebin's scale.

4. S3 Intelligent-Tiering reduces storage costs automatically. Older, infrequently accessed pastes move to cheaper tiers without manual intervention. This is the right tool for Pastebin's access pattern where most views happen in the first few days.

5. Multi-region API is a cost-benefit decision, not a default. CDN handles global content delivery. Multi-region APIs improve metadata latency by ~50ms at 2x infrastructure cost. Defer unless business requirements demand it.

6. Cleanup scales with batch operations and lifecycle rules. S3 DeleteObjects handles 1,000 deletions per API call. S3 lifecycle rules provide automatic cleanup as a safety net. Together they keep storage costs controlled without complex distributed cleanup infrastructure.

Exercise: Exercise 1: Calculate the monthly CloudFront bill at 10x scale. Data transfer: 2,500 reads/sec × 10 KB (3 KB compressed) × 3600 × 24 × 30 = ? GB/month. At $0.085/GB for the first 10 TB, what's the monthly cost? Is it cheaper to serve directly from S3?

Exercise 2: Design a monitoring dashboard for Pastebin at 10x scale. What are the top 5 metrics you'd track? (Hint: CDN hit rate, cache hit rate, S3 storage growth, P99 latency, cleanup lag.) For each metric, define the alert threshold that triggers investigation.

Exercise 3: The company acquires a competitor and needs to migrate 50 TB of paste content from the competitor's storage to your S3. Design the migration plan. How do you handle conflicting paste IDs? How do you migrate without downtime? What's the estimated migration time and cost?

Back to System Design Interviews