System design interview question
Design a Real-Time Event Streaming Analytics Platform with Pluggable Processors
Design a system that ingests high-volume events from multiple heterogeneous sources (Kafka, HTTP webhooks, database CDC), applies composable transformation and aggregation pipelines, and routes enriched events to multiple sinks (data warehouses, real-time dashboards, notification systems) with exactly-once semantics and backpressure handling. Key entities include event sources with varying schemas, stateful processors that must handle late/out-of-order data, aggregation windows (tumbling, sliding, session-based), and sinks with different failure modes. Critical challenges: designing an extensible processor abstraction that avoids tight coupling, managing mutable state across distributed stream workers without race conditions, implementing fault-tolerant checkpointing across heterogeneous sink types, and gracefully handling scenarios like processor scaling, topology changes, and poison pill events that crash specific workers.
Practice this question step by step with an AI interviewer that reviews your requirements, your architecture and your trade-offs, and scores the design.