System design interview question
Design a Distributed Multi-Tenant API Gateway and Request Routing System like Kong Enterprise at Scale
Design a high-performance, multi-tenant API gateway that routes and transforms millions of requests per second across heterogeneous backend services. The system must support 50M+ API consumers, 10K+ tenant organizations, 100K+ upstream services, with sub-50ms latency p99, 99.99% availability, and handle traffic spikes during peak hours. Tenants require fine-grained routing policies (path-based, header-based, weighted canary deployments), request/response transformation, authentication/authorization, rate limiting per tenant, distributed tracing, and graceful degradation under failure. The gateway must be horizontally scalable, support zero-downtime configuration updates, and provide strong isolation between tenants while minimizing resource overhead per tenant.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.