System design interview question
Design a Distributed Machine Learning Model Registry and Serving Platform for a Global AI Inference Network (Like Hugging Face Model Hub at Enterprise Scale)
Design a system that allows data science teams across 50+ global offices to register, version, discover, and serve machine learning models at scale. The platform must support 100M+ daily inference requests across 10K+ unique models, with latencies under 200ms for real-time predictions. Models range from 50MB to 100GB, and teams require instant model rollback, A/B testing across model versions, and per-user model access control. The system must handle 50K concurrent inference requests, support multi-region replication for compliance, and provide audit trails for all model deployments and predictions.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.