System design interview question
Design ChatGPT / LLM Serving Service
Design a scalable LLM inference service with streaming tokens, context management, and GPU auto-scaling.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.