Free lesson · ML System Design Interviews

Summary & Variations

Medium9 min readmachine-learningml-system-designinterview-prep

Key Decisions Recap

System Architecture Summary

[User Opens App]
    ↓
[Serving API] ← (multi-instance, load balanced)
    ├─→ [1. Candidate Generation] (20ms)
    │   ├─ User's followers
    │   ├─ Trending posts
    │   ├─ Interest-based search
    │   └─ Collaborative filtering
    │   Result: ~500 posts
    │
    ├─→ [2. Feature Engineering] (30ms)
    │   ├─ User features (from Redis)
    │   ├─ Post features (from Redis)
    │   ├─ Context features (at request)
    │   └─ Interaction features (from HBase)
    │   Result: ~500 feature vectors
    │
    ├─→ [3. Ranking Model] (50ms)
    │   ├─ Load XGBoost model
    │   ├─ Score each post
    │   └─ Sort by score
    │   Result: ranked 500 posts
    │
    ├─→ [4. Filtering] (30ms)
    │   ├─ Remove duplicates/near-dups
    │   ├─ Balance topics
    │   ├─ Boost new creators
    │   └─ Insert ads
    │   Result: 50 final posts
    │
    └─→ [Response] (20ms network)
        └─ Return 50 posts to user

[User Interacts]
    ↓
[Impression Logs] (5M/sec stream)
    ├─ Timestamp, user_id, post_id, action
    └─ Stored in data lake
    ↓
[Daily Batch Job]
    ├─ Collect 24h of impressions
    ├─ Engineer features
    ├─ Label (did user engage?)
    └─ Train new model
    ↓
[A/B Test] (Shadow → 5% → 100%)
    └─ Compare to baseline
    ↓
[Deploy if Better]

Variations: What If?

Variation 1: Adding Video Content

Challenge: Video engagement is different. Users might watch for 10 seconds (not "engagement" yet) or 5 minutes (very engaged).

Changes:

1. New labels:

   • Don't count clicks; use watch time
   • Positive label: watched >= 30 seconds
   • Instead of engagement in {0, 1}, use watch_time_seconds

2. New features:

   • video_duration_seconds
   • video_thumbnail_quality
   • user_video_watch_history (avg watch %)
   • video_type (short-form, long-form)

3. New model:

   • Multi-task: predict both engagement (text/image) and watch_time (video)
   • Or separate model for video ranking

4. New metrics:

   • For video: average watch completion rate (%)
   • For text: CTR

Code sketch:

def get_video_engagement_label(impression):
    """For videos, use watch time, not click."""
    video_session = get_video_session(
        user_id=impression.user_id,
        post_id=impression.post_id
    )
    watch_time = video_session['watch_duration_seconds']

    if watch_time >= 30:  # Watched at least 30 seconds
        return 1
    else:
        return 0

# Multi-task model
model = MultiTaskModel()
# For text/images: predict P(engagement)
# For videos: predict watch_time (regression)
predictions = model.predict(features)
# Result: (p_engagement, predicted_watch_time_seconds)

Variation 2: Handling Filter Bubbles & Misinformation

Challenge: Feed optimization can create echo chambers. Users only see content confirming their views, leading to polarization.

Mitigation:

1. Diversity constraints:

   • Reserve 20% of feed for "diverse perspectives"
   • Explicitly surface posts disagreeing with user (but from reputable sources)

2. Quality signals:

   • Downweight posts from unreliable sources
   • Boost fact-checked posts
   • Track misinformation_flag in post metadata

3. Metrics:

   • Track feed diversity score (topic, author diversity)
   • Measure user exposure to counter-opinions
   • Monitor polarization metrics

Code sketch:

def rank_with_diversity(candidates, user_id, model):
    """Include diversity in ranking."""

    scores = model.predict(candidates)
    ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)

    final_feed = []
    topics_shown = {}
    reliability_score_sum = 0

    for post_id, score in ranked:
        post = fetch_post(post_id)
        topic = post['topic']
        reliability = post.get('reliability_score', 0.5)

        # Constraint 1: Don't oversaturate single topic
        if topics_shown.get(topic, 0) > len(final_feed) * 0.3:
            continue

        # Constraint 2: Ensure minimum diversity
        diversity_slot = len(final_feed) % 5  # Every 5th post
        if diversity_slot == 4:  # Fill with diverse/counter perspective
            if reliability > 0.7:  # But still credible
                final_feed.append(post_id)
        else:
            final_feed.append(post_id)

        topics_shown[topic] = topics_shown.get(topic, 0) + 1

        if len(final_feed) == 50:
            break

    return final_feed

def compute_feed_diversity(feed_posts):
    """Measure diversity of feed."""
    topics = [fetch_post(p)['topic'] for p in feed_posts]
    authors = [fetch_post(p)['author_id'] for p in feed_posts]

    topic_diversity = len(set(topics)) / len(topics)
    author_diversity = len(set(authors)) / len(authors)

    return (topic_diversity + author_diversity) / 2

Variation 3: Multi-Language Support

Challenge: Posts in different languages. Users in Mexico speak Spanish; users in India speak English, Hindi, Tamil, etc.

Changes:

1. Language detection:

   • Add post_language feature
   • Include user's preferred_languages (list of ISO-639 codes)

2. Language-matching feature:

   features['user_speaks_post_language'] = 1 if post_language in user_languages else 0
   

3. Multilingual embeddings:

   • Use multilingual BERT instead of English-only
   • Post embeddings capture meaning across languages

4. Culture-aware engagement:

   • Some cultures engage more with video (India); others with text (Japan)
   • Could have language-specific models

Code sketch:

from transformers import AutoTokenizer, AutoModel

# Use multilingual embeddings
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-multilingual-cased")
model = AutoModel.from_pretrained("distilbert-base-multilingual-cased")

def get_multilingual_embedding(text, language):
    """Encode post in any language."""
    tokens = tokenizer(text, return_tensors='pt', max_length=512)
    embeddings = model(**tokens).last_hidden_state.mean(dim=1)
    return embeddings[0].detach().numpy()

# Feature engineering
features['post_embedding'] = get_multilingual_embedding(post_text, post_language)
features['user_speaks_language'] = int(post_language in user.preferred_languages)

Decision Cards (Quick Reference)

When to Use XGBoost vs Neural Networks

╔═════════════════════════════════════════════════════╗
║           XGBoost vs Neural Networks                ║
╠═════════════════════════════════════════════════════╣
║                                                     ║
║ Use XGBoost if:                                     ║
║ • Tabular features (no embeddings)                  ║
║ • <10k features                                     ║
║ • Need fast inference (must be <10ms)               ║
║ • Want interpretable feature importance             ║
║ • Team is less experienced in deep learning        ║
║                                                     ║
║ Use Neural Networks if:                             ║
║ • Have embeddings (images, text, audio)             ║
║ • >10k features or very sparse features             ║
║ • Can afford slower inference (50-100ms OK)         ║
║ • Have large data (>100M samples)                   ║
║ • Want state-of-the-art accuracy                    ║
║                                                     ║
║ For Personalized Feed: XGBoost is usually better   ║
║ (unless you have video/image embeddings)           ║
║                                                     ║
╚═════════════════════════════════════════════════════╝

Handling Data Imbalance

╔═════════════════════════════════════════════════════╗
║         Data Imbalance Solutions                    ║
╠═════════════════════════════════════════════════════╣
║                                                     ║
║ Problem: 95% negative, 5% positive examples        ║
║                                                     ║
║ [1] Downsampling (Fast)                             ║
║     └─ Randomly drop 90% of negatives               ║
║     └─ Train on balanced 50-50 dataset              ║
║     └─ Con: Throw away data                         ║
║                                                     ║
║ [2] Upweighting (Recommended)                       ║
║     └─ Give positive examples weight 20x            ║
║     └─ Keep all data                                ║
║     └─ Model learns from full signal                ║
║                                                     ║
║ [3] Threshold Tuning (Simple)                       ║
║     └─ Train normally                               ║
║     └─ Lower decision threshold (0.5 → 0.2)         ║
║     └─ Increase recall/sensitivity                  ║
║                                                     ║
║ For Feed: Use [2] (Upweighting) or [1] then [3]   ║
║                                                     ║
╚═════════════════════════════════════════════════════╝

Real-Time vs Batch Ranking

╔═════════════════════════════════════════════════════╗
║        Real-Time vs Batch Ranking                   ║
╠═════════════════════════════════════════════════════╣
║                                                     ║
║ Real-Time Ranking:                                  ║
║ ✓ Fresh signals (latest likes, comments)           ║
║ ✓ Viral posts captured immediately                 ║
║ ✗ Slow (must score all 500M posts)                 ║
║ ✗ Expensive (compute per user per request)         ║
║                                                     ║
║ Batch Ranking (Pre-computed):                       ║
║ ✓ Fast (<1ms)                                       ║
║ ✓ Cheap (compute once, reuse)                       ║
║ ✗ Stale (generated 5-30 min ago)                   ║
║ ✗ Can't adapt to user behavior change               ║
║                                                     ║
║ Hybrid (Recommended):                               ║
║ • Batch: Pre-compute top-100 feeds per user         ║
║ • Real-time: Refresh every 5 minutes                ║
║ • Serve from cache (99.9% hit rate)                 ║
║ • If not cached: fall back to real-time ranking     ║
║                                                     ║
╚═════════════════════════════════════════════════════╝

Cold Start Strategies

╔═════════════════════════════════════════════════════╗
║            Cold Start Solutions                     ║
╠═════════════════════════════════════════════════════╣
║                                                     ║
║ New User (no history):                              ║
║ • Ask for 3-5 interests during signup               ║
║ • Show trending posts + popular creators            ║
║ • Use implicit signals: clicks, hover time          ║
║ • After 10 actions, switch to personalized          ║
║                                                     ║
║ New Post (no engagement):                           ║
║ • Use author popularity (follower count)            ║
║ • Use topic popularity (trending tags)              ║
║ • Show to small sample of interested users          ║
║ • Learn from impressions, not just clicks           ║
║ • Gradually increase visibility as engagement grows ║
║                                                     ║
║ New Creator (no history):                           ║
║ • Boost diversity: reserve 5% of feed                ║
║ • Show to users interested in their topic           ║
║ • Track their engagement rate separately            ║
║ • Reward quality, not just popularity                ║
║                                                     ║
╚═════════════════════════════════════════════════════╝

Practice Questions

Test your understanding with these questions:

1. Ranking Quality

   • Why is NDCG better than AUC for ranking? (Hint: position matters)
   • How would you handle a model that ranks low-quality posts high in engagement?

2. Bias & Fairness

   • What's position bias? How does it affect training?
   • How would you detect filter bubbles in your feed system?

3. Data & Labeling

   • Why do we need to wait 24 hours before labeling? What could go wrong if we label immediately?
   • How would you handle posts that go viral weeks after posting?

4. Scale & Performance

   • If you have 100k QPS and each request takes 100ms, how many servers do you need?
   • How would you optimize if latency is 300ms but target is 200ms?

5. Online & Offline

   • Your offline AUC improves 0.75 → 0.77, but CTR drops 8% → 7.5% in A/B test. What happened?
   • How would you measure the business impact of a 1% CTR increase?

6. Feature Engineering

   • Why is post age an important feature? How would you encode it?
   • What's the difference between interaction features and context features?

7. Real-World Trade-offs

   • You can optimize for clicks or time spent. Which would you choose and why?
   • How would you balance creator fairness with engagement maximization?

Related Case Studies

Once you master Personalized Feed, explore these related systems:

Key Takeaways

For Interviews

1. Start with clarity: Ask clarifying questions about scale, users, business goals
2. Think in stages: Two-stage ranking is standard for large-scale systems
3. Data is everything: Spend time on features, labeling, and handling bias
4. Measure online: Offline metrics mislead; A/B tests tell the truth
5. Monitor relentlessly: Feature drift, engagement decay, feedback loops happen silently

For Implementation

1. Start simple: Logistic regression baseline before neural networks
2. Use what exists: XGBoost solves 80% of ranking problems
3. Iterate fast: Small improvements (1-2% CTR lift) compound over time
4. Cache everything: Latency kills user experience; pre-compute what you can
5. Automate labeling: Manual feedback loops can't scale

For Growth

As your system grows, optimize for:

• Week 1: Get something working (XGBoost baseline, simple features)
• Week 2-4: Improve offline metrics (add more features, tune hyperparams)
• Month 2: Run A/B tests (measure real impact)
• Month 3+: Optimize for scale (caching, inference optimization, monitoring)

Additional Resources

Reading:

• "Recommenders at Scale" by Netflix
• "Addressing Bias in Machine Learning" by Fairness and Explainability papers
• XGBoost paper: "XGBoost: A Scalable Tree Boosting System"

Tools:

• XGBoost (Python)
• TensorFlow / PyTorch (deep learning)
• Spark (distributed training)
• Presto / Druid (real-time querying)

Summary Callouts

💡 Tip: In interviews, emphasize the business goal first, then translate to ML. This shows maturity.

📝 Note: Most production ML systems use XGBoost or similar tree ensembles. Not sexy, but it works.

⚠️ Watch out: A metric that improved offline might hurt online (e.g., CTR up, but session length down). Always validate online.

🎯 Interviewer's Perspective: Wrap up by discussing what you'd do next: A/B test results, monitoring strategy, and longer-term vision. Shows systems thinking.

✅ Key Insights:

• Scale requires two stages: candidate generation + ranking
• XGBoost is the production standard (interpretable, fast, expressive)
• Offline metrics are necessary but insufficient; A/B tests are ground truth
• Position bias, feedback loops, and cold start are real challenges
• Monitoring and drift detection are as important as model accuracy

Word count: ~1800 words

Full Case Study Stats

How to Use This Case Study

1. Learn: Read through all 5 files sequentially
2. Understand: Answer the practice questions
3. Discuss: Use the interview format to practice explaining concepts
4. Build: Implement a small version (candidate gen + simple ranking)
5. Interview: Use these materials to prepare for system design interviews

Back to ML System Design Interviews