Free lesson · ML System Design Interviews

Overview: The Five-Step Framework

Easy6 min readmachine-learningml-system-designinterview-prep

When you walk into an ML system design interview, you need a mental framework to guide your thinking. Without one, you'll bounce around randomly and confuse yourself (and the interviewer).

Here's our five-step framework that works for almost every ML system design problem:

The Five Steps at a Glance

🔍 Step 1: The Detective Phase - Clarify Requirements

What: Ask clarifying questions to understand the business problem, constraints, and scale.

Why it matters: This is the most important step. A wrong problem solved perfectly is still wrong.

Questions you'll ask:

• What's the business goal?
• Who are the users and what's their scale?
• What are the key constraints (latency, accuracy, cost)?
• What data do we have access to?

Time: 10-15 minutes

📋 Step 2: The Framing Phase - Frame the ML Problem

What: Translate the business problem into a clear ML task.

Why it matters: "Increase user engagement" is not an ML problem. "Predict whether a user will click a video" is.

What you'll do:

• Define the ML objective
• Specify inputs and outputs
• Choose the ML category (classification, regression, ranking, retrieval, etc.)
• Consider multiple framing options

Time: 5-8 minutes

📊 Step 3: The Data Phase - Data Preparation & Feature Engineering

What: Discuss where data comes from, how to prepare it, and how to create useful features.

Why it matters: "Garbage in, garbage out." The best model with bad data loses to a simple model with great data.

What you'll do:

• Identify data sources (user logs, labels, signals)
• Discuss data quality and cleaning
• Design features (feature engineering)
• Plan train/validation/test splits

Time: 8-12 minutes

🧠 Step 4: The Brain Phase - Model Development & Architecture

What: Choose and design the model architecture.

Why it matters: This is what most people think of as "ML," but it's actually just one piece of the puzzle.

What you'll do:

• Propose a baseline model (start simple!)
• Discuss more complex options if needed
• Talk about loss functions and optimization
• Consider training details (batch size, learning rate, etc.)

Time: 10-12 minutes

🚀 Step 5: The Ship-It Phase - Evaluation, Deployment & Monitoring

What: Discuss how to evaluate the model, deploy it, and keep it healthy in production.

Why it matters: A model that works offline but crashes in production is useless. A model that works but no one knows it's degrading is dangerous.

What you'll do:

• Define evaluation metrics
• Plan offline and online testing
• Discuss deployment architecture
• Talk about monitoring and retraining

Time: 8-10 minutes

The Full Picture: How the Steps Connect

┌─────────────────────────────────────────────────────────────┐
│ Step 1: Detective Phase                                     │
│ "What are we solving? For whom? With what constraints?"     │
└──────────────────┬──────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────────┐
│ Step 2: Framing Phase                                       │
│ "What's the ML problem? Inputs? Outputs? ML category?"      │
└──────────────────┬──────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────────┐
│ Step 3: Data Phase                                          │
│ "Where's the data? How do we prepare it? What features?"    │
└──────────────────┬──────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────────┐
│ Step 4: Brain Phase                                         │
│ "What model? What loss? How to train?"                      │
└──────────────────┬──────────────────────────────────────────┘
                   │
                   ▼
┌─────────────────────────────────────────────────────────────┐
│ Step 5: Ship-It Phase                                       │
│ "How to evaluate? Deploy? Monitor? Retrain?"                │
└─────────────────────────────────────────────────────────────┘

The Five Steps in Action: Recommend Movies

Let's walk through a quick example to see how these steps work together.

Problem statement: "Design a system to recommend movies to users on a streaming platform."

Step 1: The Detective Phase

Candidate: "Before I design a solution, let me understand the problem better. What are our key goals? Are we trying to maximize engagement, retention, or both?"

Interviewer: "Great question. Mainly engagement—we want users to click and watch our recommendations."

Candidate: "How many users do we have?"

Interviewer: "About 100 million monthly active users."

Candidate: "And what's our latency requirement? How quickly do we need to generate recommendations?"

Interviewer: "Under 200ms for the homepage. The recommendations need to appear in the feed instantly."

Candidate: "Understood. One more thing—do we have user behavior data already? Click history, ratings, watch time?"

Interviewer: "Yes, we have 5 years of click data and watch time data."

What we learned: 100M users, <200ms latency, maximizing clicks, we have historical click and watch data.

Step 2: The Framing Phase

Candidate: "So the business goal is engagement through recommendations. The ML problem I'd frame this as: predict the probability that a user will click on a movie recommendation. Inputs are the user's history and movie features. Output is a score between 0-1. This is a ranking problem—we generate top-N recommendations ranked by click probability."

Interviewer: "Good. Could we frame it differently?"

Candidate: "Sure. We could do collaborative filtering where similar users watch similar movies. Or we could predict watch time instead of clicks. But click prediction seems most aligned with our engagement goal."

What we know now: The ML task is click prediction (classification/ranking), and we have a baseline framing.

Step 3: The Data Phase

Candidate: "For data, we'd use historical user-movie interactions. Our features would include: user features (watch history, preferences), movie features (genre, rating, popularity), and interaction features (how long ago they watched similar movies). We'd split the data chronologically—train on history before a certain date, validate on recent history, test on the most recent data."

Interviewer: "Why chronological split?"

Candidate: "Because in a real system, we're always predicting the future. If we randomly split data, we'd overestimate performance—the model would train on data from the future. Chronological split is more realistic."

What we designed: Data sources, feature engineering strategy, and a realistic evaluation approach.

Step 4: The Brain Phase

Candidate: "For the model, I'd start with a simple baseline: logistic regression on engineered features. If that doesn't hit our accuracy targets, I'd explore gradient boosting or neural networks. Given our latency constraint of 200ms, we need fast inference, so deep learning would only work if we can optimize it heavily."

Interviewer: "What loss function?"

Candidate: "Binary cross-entropy since we're predicting click probability (0 or 1). We might also weight the loss—positive clicks might be rare, so we'd upweight them to prevent the model from always predicting 'no click.'"

What we chose: Baseline model with clear trade-offs (simple vs. accurate) and a loss function suited to the problem.

Step 5: The Ship-It Phase

Candidate: "For evaluation, offline we'd measure precision, recall, and AUC on the test set. Online, we'd A/B test the recommendation system—half of users see our new recommendations, half see the old system. We'd measure click-through rate, watch time, and user retention. For deployment, we'd use a model server to generate predictions in real-time. We'd monitor for drift—if user behavior changes, the model needs retraining. Maybe monthly or on-demand if performance drops."

Interviewer: "What if the model performs great offline but poorly online?"

Candidate: "Good question. That usually means our offline metrics don't capture something real. Maybe our features leak information about future clicks. Or maybe online users behave differently than our training data. We'd investigate the data distribution, adjust features, or collect more recent training data."

What we planned: Full evaluation strategy, real-time serving architecture, and monitoring/retraining approach.

Time Allocation: The Breakdown

Here's the recommended time for a 45-minute interview:

Key insight: Don't spend equal time on each step! Steps 1, 3, and 4 deserve the most attention. Step 2 and 5 are shorter but critical for showing you understand the full system.

When to Spend More / Less Time

Spend MORE time on:

• Step 1 (Detective Phase): If the problem is vague or new to you. Better to ask 20 questions than to build the wrong solution.
• Step 3 (Data Phase): If data quality or availability seems like a constraint.
• Step 4 (Brain Phase): If the interviewer drills into model trade-offs or asks you to handle specific challenges.

Spend LESS time on:

• Step 2 (Framing Phase): Once you've defined the problem clearly, keep moving. You might come back to this if the interviewer challenges your framing.
• Step 5 (Ship-It Phase): In an interview, high-level thinking is usually enough. You won't code a full serving system.

💡 Tip: If you find yourself in a deep discussion about neural network architectures in minute 30 and haven't talked about deployment, gently steer back: "That's a great point on model complexity. But before we go too deep—I want to make sure we're thinking about the latency constraints. That might actually simplify our model choice. Should we discuss serving first?"

The Framework is a Guide, Not a Straitjacket

You don't have to follow these steps in lockstep order. Sometimes an interviewer will ask you about deployment (Step 5) before you've finished data discussion (Step 3). That's fine! The framework is a mental checklist, not a rigid sequence.

But the five-step mindset should always be there. Every interview, you're answering:

1. What problem are we solving?
2. How do we define success in ML terms?
3. What data powers our system?
4. What model makes sense?
5. How does it work in the real world?

Answer those five questions well, and you've nailed the interview.

Back to ML System Design Interviews