System design interview question
Design a Multi-Modal RAG (Retrieval-Augmented Generation) System with Real-Time Index Updates and Query-Time Routing like Perplexity AI at Scale
Design a production-grade RAG system that serves 50M monthly active users with 100K concurrent queries per second, supporting both text and multi-modal (image, document) document ingestion. The system must handle sub-500ms end-to-end latency (query retrieval LLM generation response), dynamically route queries across heterogeneous LLM endpoints (GPT-4, Claude, local models), perform real-time index updates from streaming data sources (news feeds, APIs), and implement sophisticated guardrails including hallucination detection, source attribution, and content filtering. Design the retrieval layer to support hybrid search (semantic + BM25 + metadata filtering), multi-stage ranking (retrieve 1000 candidates rerank to 10), and handle cost optimization across inference providers while maintaining 99.95% availability across 5+ geographic regions.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.