Free lesson · System Design Interviews

Must-Know Concepts - The Interview Checklist

Easy11 min readsystem-designinterview-prep

Let me tell you what the checklist is not. It's not a textbook chapter. It's not a deep dive into distributed systems theory. If you want that, take our System Design Fundamentals course - that's where you learn how things work.

This lesson is the pre-flight checklist. It's what you review the night before your interview, the morning of, and in the car on the way there. Every concept here is something I've seen come up in system design interviews - not once, but repeatedly. If you walk in without knowing any one of these cold, you're gambling that it won't come up. And that's a gamble you'll eventually lose.

I'm organizing these by how frequently they appear in interviews and how catastrophic it is if you don't know them. The top of the list is "you will almost certainly be asked about this." The bottom is "it comes up in maybe one in five interviews, but if it does and you blank, it's a red flag."

Tier 1: Must Know Cold - These Come Up Every Time

Horizontal vs Vertical Scaling

Vertical scaling means adding more resources (CPU, RAM, disk) to a single machine. Horizontal scaling means adding more machines.

graph LR
    subgraph "Vertical Scaling"
        V1[Small Server<br/>4 CPU, 8GB RAM] --> V2[Big Server<br/>64 CPU, 256GB RAM]
    end

    subgraph "Horizontal Scaling"
        H1[Server 1] 
        H2[Server 2]
        H3[Server 3]
        H4[Server N...]
        LB[Load Balancer] --> H1
        LB --> H2
        LB --> H3
        LB --> H4
    end

This diagram compares vertical and horizontal scaling. Vertical scaling upgrades a single server from small to large. Horizontal scaling adds multiple servers behind a load balancer.

The interview answer is almost always horizontal scaling. Why? Because vertical scaling has a ceiling - you can't make a single machine infinitely powerful. Horizontal scaling has no theoretical ceiling. But horizontal scaling introduces complexity: state management, consistency, network partitions, coordination overhead.

Interview: What to say: "I'd scale horizontally because [the workload/traffic/data] exceeds what a single machine can handle. The tradeoff is that horizontal scaling requires us to handle distributed state - but for a system at this scale, that complexity is unavoidable."

Load Balancing

A load balancer distributes incoming requests across multiple servers. You should know three things: why (no single server can handle all traffic), how (round-robin, least connections, consistent hashing), and where (between clients and web servers, between web servers and application servers, between application servers and databases).

Common algorithms:

Algorithm	How It Works	Best For
Round Robin	Rotate through servers in order	Stateless services with similar capacity
Least Connections	Send to server with fewest active connections	Variable-length requests
IP Hash	Hash the client IP to pick a server	Session stickiness without cookies
Weighted Round Robin	Like round robin, but some servers get more traffic	Heterogeneous server fleet
Consistent Hashing	Hash-ring-based distribution	Distributed caches and databases

CAP Theorem

The CAP theorem states that a distributed system can provide at most two of three guarantees: Consistency (every read receives the most recent write), Availability (every request receives a response), and Partition Tolerance (the system continues operating during network partitions).

In practice, network partitions will happen - so the real choice is between consistency and availability during a partition. This is not an abstract concept. This is a question that comes up in almost every system design interview.

graph TD
    CAP[CAP Theorem] --> CP[CP Systems<br/>Choose Consistency]
    CAP --> AP[AP Systems<br/>Choose Availability]
    
    CP --> CP1[Examples:<br/>HBase, MongoDB,<br/>Redis Cluster]
    CP --> CP2[Use when:<br/>Financial data,<br/>inventory counts,<br/>user auth]
    
    AP --> AP1[Examples:<br/>Cassandra, DynamoDB,<br/>CouchDB]
    AP --> AP2[Use when:<br/>Social feeds,<br/>analytics, caching,<br/>session data]

This diagram shows the CAP theorem split. CP systems choose consistency and include HBase, MongoDB, and Redis Cluster, and are used for financial data, inventory counts, and authentication. AP systems choose availability and include Cassandra, DynamoDB, and CouchDB, and are used for social feeds, analytics, and caching.

Interview: What to say: "For this use case, I'd lean toward [CP/AP] because [consistency matters more than availability / availability matters more than consistency] here. For example, in a payment system, I can't have two servers disagree about whether a charge went through - that's a CP requirement. But for a social media timeline, showing a slightly stale feed is fine - that's an AP scenario."

Consistency Models

Beyond CAP, you need to know the spectrum of consistency:

Strong consistency: Every read sees the most recent write. Simplest to reason about, hardest to scale.
Eventual consistency: Reads may return stale data, but all replicas will converge to the same value eventually. Easiest to scale, hardest to reason about.
Causal consistency: If event A causes event B, everyone sees A before B. A middle ground.
Read-your-writes consistency: You always see your own most recent write. Common in user-facing applications.

Database Replication

Replication copies data across multiple machines for durability and read scalability. You should know:

Leader-follower (master-slave): One leader handles writes; followers replicate and handle reads. Simple but the leader is a bottleneck.
Leader-leader (multi-master): Multiple nodes accept writes. Higher availability but conflict resolution is hard.
Leaderless: Any node accepts reads and writes. Uses quorum reads/writes (read from R nodes, write to W nodes, where R + W > N). Cassandra and DynamoDB use this.

Database Sharding (Partitioning)

Sharding splits data across multiple databases so no single database holds everything. You should know:

Range-based sharding: Split by ranges (users A-M on shard 1, N-Z on shard 2). Simple but prone to hotspots.
Hash-based sharding: Hash the partition key and distribute by hash. Even distribution but loses range query ability.
Consistent hashing: A hash ring that minimizes data movement when adding or removing shards.

Warning: The sharding question you must be ready for: "How do you handle a query that spans multiple shards?" This is where candidates stumble. The answer involves scatter-gather (query all shards, merge results), or denormalization (store data redundantly so queries hit a single shard), or a separate indexing layer (like Elasticsearch) that indexes across shards. Know all three options and when each fits.

Caching

Caching stores frequently accessed data in memory for faster retrieval. You should know:

Cache-aside (lazy loading): Application checks cache first. If miss, fetch from database, store in cache. Most common pattern.
Write-through: Every write goes to both cache and database. Cache is always up to date, but writes are slower.
Write-behind (write-back): Write to cache only; cache asynchronously writes to database. Fast writes, risk of data loss.
Cache invalidation: TTL-based (expire after N seconds), event-based (invalidate on write), or manual. Cache invalidation is famously hard - Phil Karlton's quote about the two hard things in computer science.

Message Queues and Asynchronous Processing

When you can't process a request synchronously (it takes too long, it requires multiple services, it might fail), you put it on a queue and process it later. You should know:

Why: Decoupling producers and consumers. Handling traffic spikes. Enabling retries. Making systems more resilient.
Delivery guarantees: At-most-once (may lose messages), at-least-once (may duplicate), exactly-once (hard to achieve - usually at-least-once + idempotency).
Technologies: Kafka (log-based, high throughput, ordered), RabbitMQ (traditional broker, flexible routing), SQS (managed, simple), Redis Streams (lightweight).

sequenceDiagram
    participant C as Client
    participant S as API Server
    participant Q as Message Queue
    participant W as Worker
    participant DB as Database

    C->>S: POST /order
    S->>Q: Enqueue order event
    S->>C: 202 Accepted
    Q->>W: Dequeue order event
    W->>DB: Process and store order
    W->>Q: Acknowledge completion

This sequence diagram shows asynchronous processing via a message queue. The client sends a POST request to the API server. The server enqueues the order event and immediately returns a 202 Accepted response. A worker dequeues the event, processes it, stores it in the database, and acknowledges completion.

Tier 2: Must Know Well - These Come Up Frequently

Content Delivery Networks (CDNs)

CDNs cache static content (images, videos, CSS, JavaScript) at edge locations geographically close to users. They reduce latency by serving content from the nearest edge node instead of the origin server. You should know push CDN (origin pushes content to edges) vs pull CDN (edges fetch from origin on cache miss). Pull CDNs are more common.

DNS (Domain Name System)

DNS resolves domain names to IP addresses. In system design, DNS is used for load balancing (DNS round-robin), geographic routing (route users to the nearest data center), and failover (change DNS records to redirect traffic away from a failed server). Know that DNS has propagation delays - changes don't take effect instantly.

API Gateway

An API gateway sits between clients and backend services. It handles routing, rate limiting, authentication, SSL termination, and request/response transformation. In microservices architectures, the API gateway is the single entry point. Examples: AWS API Gateway, Kong, Nginx.

Rate Limiting

Rate limiting restricts the number of requests a client can make in a given time window. It protects against abuse, ensures fair usage, and prevents cascading failures. Common algorithms: token bucket, sliding window log, fixed window counter, sliding window counter. Know at least one well enough to implement it.

Reverse Proxy

A reverse proxy sits between clients and servers. It handles load balancing, SSL termination, caching, and compression. Nginx and HAProxy are the standard examples. The key distinction from a forward proxy: a forward proxy acts on behalf of clients; a reverse proxy acts on behalf of servers.

Database Indexing

Indexes speed up reads at the cost of slower writes and additional storage. B-tree indexes are the default for relational databases. Hash indexes are useful for exact-match lookups. Composite indexes cover queries on multiple columns. The interview question is almost always: "How would you optimize this query?" The answer usually involves an index.

ACID Transactions

Atomicity (all or nothing), Consistency (data satisfies constraints), Isolation (concurrent transactions don't interfere), Durability (committed data survives failures). Relational databases provide ACID. Most NoSQL databases trade some ACID properties for scalability - you should know which ones and why.

Consensus Protocols

Consensus protocols let distributed nodes agree on a value. You should know that Paxos and Raft exist and what they do (leader election, log replication) without needing to implement them. The interview relevance: when you say "the system uses a leader-follower setup," the interviewer might ask "how does the system elect a new leader if the current one fails?" The answer is a consensus protocol.

Tier 3: Should Know - These Come Up in Deep Dives

Consistent Hashing

A technique for distributing data across nodes such that adding or removing a node only requires redistributing a minimal amount of data. Used in distributed caches (Memcached), databases (DynamoDB, Cassandra), and load balancers. Know the hash ring concept and virtual nodes.

Read vs Write Optimization

Some systems are read-heavy (100:1 read/write ratio - social media timelines). Some are write-heavy (logging, analytics, IoT sensor data). The architecture for each is different. Read-heavy systems use read replicas, caching, and CDNs. Write-heavy systems use append-only logs, write-behind caching, and sharding.

Normalization vs Denormalization

Normalization eliminates data redundancy. Denormalization adds it back for query performance. In system design interviews, the tradeoff is always: "Normalization saves storage and ensures consistency, but joins are expensive at scale. Denormalization is faster for reads but requires updating multiple copies of the same data."

Blob Storage

Large binary objects (images, videos, files) should not be stored in a relational database. They go in blob storage - S3 is the standard answer. The database stores a reference (URL or key) to the blob. Know that S3 provides eventual consistency for overwrite PUTs and DELETEs, and read-after-write consistency for new PUTs.

Heartbeat and Health Checks

Mechanisms for detecting failed nodes. A heartbeat is periodic "I'm alive" signals between nodes. A health check is an external system (load balancer, orchestrator) polling a node's health endpoint. Know the difference between liveness checks (is the process running?) and readiness checks (is the process ready to serve traffic?).

Circuit Breaker Pattern

When a downstream service is failing, a circuit breaker stops sending requests to it - preventing cascade failures. Three states: closed (normal operation), open (all requests fail fast), half-open (test with a few requests to see if the service has recovered). This pattern shows up in resilience discussions.

Leader Election

How distributed nodes choose a leader. Mechanisms include Raft/Paxos consensus, ZooKeeper-based election, or database-based locking. Know when leader election is needed (database replication, task coordination) and the failure modes (split brain - two nodes both think they're the leader).

The Concept Map

Here's how these concepts relate to each other. In an interview, you'll rarely discuss one concept in isolation - they chain together.

graph TD
    SCALE[Scaling] --> HS[Horizontal Scaling]
    HS --> LB[Load Balancing]
    HS --> SHARD[Sharding]
    HS --> REP[Replication]
    
    SHARD --> CH[Consistent Hashing]
    REP --> CONS[Consistency Models]
    CONS --> CAP[CAP Theorem]
    
    PERF[Performance] --> CACHE[Caching]
    PERF --> CDN[CDN]
    PERF --> IDX[Indexing]
    
    ASYNC[Async Processing] --> MQ[Message Queues]
    MQ --> DELIVERY[Delivery Guarantees]
    
    RESIL[Resilience] --> CB[Circuit Breaker]
    RESIL --> HB[Health Checks]
    RESIL --> LE[Leader Election]
    LE --> CONSENSUS[Consensus Protocols]

This concept map shows relationships between system design concepts. Scaling connects to horizontal scaling, which leads to load balancing, sharding, and replication. Sharding connects to consistent hashing. Replication connects to consistency models and CAP theorem. Performance connects to caching, CDN, and indexing. Async processing connects to message queues and delivery guarantees. Resilience connects to circuit breakers, health checks, and leader election, which connects to consensus protocols.

How to Study This Checklist

Do not try to memorize this list. Instead, for each concept, make sure you can:

Define it in one sentence. "Consistent hashing is a technique for distributing data across nodes so that adding or removing a node requires minimal data movement."
Explain when you'd use it. "I'd use consistent hashing when I need to shard a distributed cache across many nodes and I expect the number of nodes to change over time."
Name the tradeoff. "The tradeoff with consistent hashing vs simple modular hashing is complexity - consistent hashing is more code, but simple modular hashing requires rehashing everything when nodes change."
Draw it. Can you sketch a hash ring on a whiteboard in 30 seconds?

If you can do those four things for every concept on this list, you're prepared.

Tip: The night before your interview: Go through this list top to bottom. For each concept, say the one-sentence definition out loud. If you stumble on any, spend five minutes reviewing it. This is a calibration check, not a cram session.

Six Key Takeaways

System design interviews draw from a finite set of concepts. This list covers 90% of what comes up.
Tier 1 concepts (scaling, load balancing, CAP, consistency, replication, sharding, caching, message queues) come up in almost every interview. Know them cold.
Tier 2 concepts (CDN, DNS, API gateway, rate limiting, indexing, ACID, consensus) come up frequently and are often part of deep dives.
Tier 3 concepts (consistent hashing, normalization, blob storage, circuit breaker, leader election) come up in targeted deep dives. They separate "good" from "great."
Concepts chain together. You'll rarely discuss one in isolation - "I'd shard the database using consistent hashing, with three replicas per shard for durability, and a cache layer in front for read performance." That's four concepts in one sentence.
For each concept, know the one-sentence definition, when to use it, the tradeoff, and how to draw it.

Exercises

Exercise: Exercise 1: Set a timer for 20 minutes. Without looking at this lesson, write down every system design concept you can think of. Then compare your list to this one. What did you miss?

Exercise 2: For each Tier 1 concept, practice the "one sentence definition + when to use + tradeoff" pattern. Record yourself. Are you clear and concise, or are you rambling?

Exercise 3: Pick a system design question like "Design a URL Shortener." Go through your design and label every concept from this list that you use. Did you use at least five Tier 1 concepts? If not, your design might be too shallow.

Resources

Designing Data-Intensive Applications (Martin Kleppmann) - The definitive deep reference for almost every concept on this list
System Design Primer (GitHub) - Visual summaries of core concepts with diagrams
AWS Well-Architected Framework - Amazon's framework for building reliable, scalable systems
Google SRE Book - Free Online - Google's guide to building and operating large-scale systems

Back to System Design Interviews