System design interview question
Design a Distributed Rate Limiter for API Gateway
Design a rate limiting system that prevents clients from exceeding their API quota across a distributed cluster of servers. The system must track request counts per client (identified by API key or IP) across multiple gateway nodes, enforce limits in real-time, and handle scenarios where clients are load-balanced across different servers. Key entities include clients, rate limit rules (requests per second/minute), and gateway nodes. Important edge cases: clock skew between servers, race conditions when a client's request arrives at multiple nodes simultaneously, and graceful degradation when the rate limiter service is unavailable. The system should support different limit strategies (token bucket, sliding window) and provide clear feedback when limits are exceeded.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.