Free lesson · System Design Fundamentals

Batch vs Stream Processing

Hard11 min readsystem-designhlddistributed-systems

26.1 Batch vs Stream Processing

The Fundamental Distinction

Picture this. You run an e-commerce platform. Every day, millions of customers browse products, add items to carts, and make purchases. At the end of each day, your analytics team runs a massive computation across the entire day's transaction log to produce revenue reports, identify trending products, and update recommendation models. That computation takes two hours, and the reports land in dashboards by 6 AM the next morning. That is batch processing - you collect a bounded chunk of data, process it all at once, and produce a result.

Now picture a different need at the same company. A customer places an order for a laptop. The moment that order event fires, you need to check inventory in real time, send a confirmation email within seconds, update the shipping system, and adjust the product's "items remaining" count on the page so the next customer sees accurate stock levels. You cannot wait until tomorrow's batch run to do this. You need to process each event as it arrives, continuously, with no defined "end." That is stream processing - you handle data as an unbounded, never-ending flow.

This distinction is one of the most fundamental in all of system design, and interviewers test it constantly. Not because they want you to recite definitions, but because choosing the wrong processing model for a problem leads to systems that are either painfully slow when they need to be fast, or needlessly complex when simplicity would have worked fine.

Two Mental Models for Data

The deepest way to understand batch versus stream processing is to think about how each model views data itself.

Batch processing treats data as a noun. It is a thing - a dataset, a file, a table. It has a beginning and an end. You can measure its size. You can sort it. You can load the whole thing into memory (or across a cluster) and run computations that see every record. The dataset exists before processing starts, and processing produces a new dataset as output.

Stream processing treats data as a verb. It is something happening - an event, a message, a change. There is no "whole dataset" to look at because the data never stops arriving. You process each piece as it flows past, maintaining running state that evolves over time. The stream has no natural boundary; you impose artificial ones (time windows, counts) when you need aggregate answers.

This is not just a philosophical distinction. It shapes every architectural decision downstream: how you store intermediate state, how you handle failures, how you scale, and what guarantees you can offer about result correctness.

graph LR
    subgraph Batch["Batch Processing"]
        D1["Bounded Dataset<br/>(files, tables)"] --> P1["Process<br/>All At Once"]
        P1 --> R1["Complete Result<br/>(new dataset)"]
    end

    subgraph Stream["Stream Processing"]
        D2["Unbounded Events<br/>(continuous flow)"] --> P2["Process<br/>Each Event"]
        P2 --> R2["Continuous Output<br/>(updated state)"]
        P2 --> P2
    end

    style Batch fill:#1e293b,stroke:#6366f1,color:#e2e8f0
    style Stream fill:#1e293b,stroke:#10b981,color:#e2e8f0

This diagram shows two processing models side by side. On the left, batch processing takes a bounded dataset like files or tables, processes everything at once, and produces a complete result as a new dataset. On the right, stream processing receives unbounded events as a continuous flow, processes each event individually, and produces continuous output as updated state, with a loop indicating the processing never stops.

When Batch Processing Is the Right Choice

Batch processing shines in specific situations, and understanding those situations is what separates a good system design answer from a mediocre one.

The data is naturally bounded. Some data arrives in discrete chunks. A company's quarterly financial data closes on a specific date. A day's worth of server logs has a clear start and end. When data has natural boundaries, batch processing aligns with reality rather than fighting it.

You need global computations. Some computations require seeing the entire dataset to produce a correct answer. Training a machine learning model on six months of user behavior, computing the median salary across all employees, or rebuilding a search index from scratch - these need access to the whole picture. Stream processing can approximate some of these with running aggregates, but the results are different (and sometimes meaningfully wrong) compared to seeing every record.

Latency tolerance is high. If the consumer of your output can wait hours or even a day, batch processing is almost always simpler. Daily analytics dashboards, monthly billing calculations, nightly data warehouse refreshes - these have deadlines measured in hours, not seconds. The simplicity of batch processing in these cases is a real advantage, because simpler systems break less.

Throughput matters more than latency. Batch systems can be optimized for throughput in ways stream systems cannot. When you know the entire input upfront, you can sort it, partition it, schedule resources optimally, and amortize overhead across millions of records. A batch job processing 10 billion records overnight can achieve per-record costs that would be impossible in a stream processor handling one record at a time.

Tip: The interview shortcut: if the problem says "daily report," "nightly rebuild," "weekly aggregation," or "monthly billing," batch processing is almost certainly the right starting point. Interviewers who hear you immediately consider batch for these scenarios know you understand the tradeoff space.

When Stream Processing Is the Right Choice

Stream processing earns its complexity when the problem genuinely demands it.

Low-latency requirements. When the business needs a response in seconds or milliseconds - fraud detection on credit card transactions, real-time bidding in ad systems, alerting on server errors - waiting for a batch window is not an option. The value of the computation degrades rapidly with time. A fraud alert that arrives 24 hours after the fraudulent transaction is not a fraud alert; it is a post-mortem.

Continuous data sources. Some data sources are inherently unbounded. IoT sensor readings from a factory floor, click events from a website, GPS coordinates from a fleet of delivery trucks - these do not pause at midnight to wait for a batch job. Processing them in batches means imposing artificial boundaries on naturally continuous data, which creates complexity around boundary handling (what happens to an event that straddles two batch windows?).

Derived views that must stay fresh. If your system maintains materialized views, caches, or search indexes that users expect to be current, stream processing keeps them updated continuously rather than stale-until-next-batch-run. When a seller updates a product price on an e-commerce site, customers should see the new price within seconds, not after tonight's batch refresh.

Event-driven architectures. In systems where components communicate through events (order placed, payment received, shipment dispatched), stream processing is the natural fit. Each component subscribes to relevant event streams and reacts in real time. Trying to batch-process event-driven workflows creates awkward polling loops and artificial delays.

The Tradeoff Matrix

Here is where the comparison gets practical. Every system design decision involves tradeoffs, and interviewers want to hear you reason through them explicitly.

quadrantChart
    title Choosing Your Processing Model
    x-axis Low Latency Need --> High Latency Need
    y-axis Bounded Data --> Unbounded Data
    quadrant-1 Batch (classic fit)
    quadrant-2 Batch (consider micro-batch)
    quadrant-3 Stream (classic fit)
    quadrant-4 Stream (with windowing)
    Daily reports: [0.85, 0.8]
    ML model training: [0.9, 0.7]
    Fraud detection: [0.15, 0.25]
    Real-time dashboards: [0.2, 0.3]
    Log aggregation: [0.6, 0.4]
    IoT monitoring: [0.15, 0.15]

This quadrant chart helps choose a processing model based on two axes. The horizontal axis ranges from low latency need on the left to high latency need on the right. The vertical axis ranges from unbounded data at the bottom to bounded data at the top. The top-right quadrant labeled Batch classic fit contains daily reports and ML model training. The bottom-left quadrant labeled Stream classic fit contains fraud detection, real-time dashboards, and IoT monitoring. Log aggregation sits in the middle, suggesting either model could work.

The Spectrum Between Batch and Stream

Here is something that trips up a lot of candidates: batch and stream processing are not a binary choice. There is a spectrum between them, and several hybrid approaches occupy the middle ground.

Micro-batching collects events into very small batches - say, every 500 milliseconds or every 100 events - and processes each micro-batch as a unit. Spark Streaming uses this approach. You get near-real-time latency (sub-second) with much of the simplicity of batch processing (each micro-batch is a bounded dataset). The tradeoff is that your minimum latency is bounded by the micro-batch interval; you can never respond faster than that window.

Windowed stream processing applies batch-like computations to windows of streaming data. A five-minute tumbling window, for example, collects all events within each five-minute period and computes aggregates over them. This gives you stream-like freshness with batch-like aggregate computations. But windowing introduces its own complexity: what happens to late-arriving events? How do you handle events that belong to a window that has already closed?

Change Data Capture (CDC) turns batch data sources (databases) into streams. Instead of periodically dumping an entire database table, CDC captures each insert, update, and delete as an event in a stream. This bridges the gap between systems that store data in batch-oriented formats and systems that need to react to changes in real time.

graph LR
    B["Pure Batch<br/>Hours latency"] --> MB["Micro-Batch<br/>Seconds latency"]
    MB --> WS["Windowed Stream<br/>Sub-second"]
    WS --> PS["Pure Stream<br/>Per-event"]

    style B fill:#1e293b,stroke:#6366f1,color:#e2e8f0
    style MB fill:#1e293b,stroke:#8b5cf6,color:#e2e8f0
    style WS fill:#1e293b,stroke:#a78bfa,color:#e2e8f0
    style PS fill:#1e293b,stroke:#10b981,color:#e2e8f0

This diagram shows the spectrum from pure batch processing on the left, through micro-batch, then windowed stream processing, to pure stream processing on the right. Latency decreases from hours to per-event as you move left to right along the spectrum.

Correctness and the Re-processability Advantage

One of batch processing's most underappreciated strengths is re-processability. If a batch job produces wrong results - maybe there was a bug in the transformation logic, or the input data had corruption - you fix the code and rerun the job on the same input. The input is still there, sitting in files or tables, unchanged. You get a second chance.

Stream processing does not naturally offer this. Once an event flows past your processor, it is gone unless you have explicitly stored it somewhere. If your stream processing logic had a bug, the events it processed incorrectly are already downstream, and correcting them requires either replaying from a durable log (like Kafka) or building separate reconciliation pipelines.

This is why many production architectures keep the raw event log even when they process events in real time. The log serves as the "batch input" for corrections, backfills, and audits. You get the low latency of stream processing for the happy path, and the correctability of batch processing when things go wrong.

Interview: When an interviewer asks "how would you handle a bug in your processing logic?" - this is the moment to demonstrate that you understand the re-processability tradeoff. For batch systems, you say "rerun the job." For stream systems, you say "replay from the durable log, using consumer offsets to reprocess from the point of the bug." If your stream system does not have a durable log, acknowledge that as a design limitation.

Real-World Patterns: Both Models Together

In practice, most large-scale systems use both batch and stream processing for different parts of the same problem. Here are common patterns you will encounter in interviews.

Pattern 1: Stream for operational, batch for analytical. An e-commerce platform processes orders in real time (stream) for fulfillment, but runs nightly batch jobs to compute daily revenue, product rankings, and recommendation model updates. The two pipelines share the same underlying data but serve different consumers with different latency requirements.

Pattern 2: Stream as primary, batch as correction. A real-time ad bidding system processes bid requests as a stream, making decisions in under 50 milliseconds. But a nightly batch job reconciles actual impressions against bids, correcting for events that were dropped, duplicated, or misattributed in the stream path. The batch job is the source of truth; the stream is the fast-but-approximate path.

Pattern 3: Batch to bootstrap, stream to maintain. When building a search index, you might run a one-time batch job to index the entire existing catalog (millions of products). Once the index is built, a stream processor handles incremental updates - new products added, prices changed, items removed - keeping the index fresh without ever needing another full rebuild.

Important: In system design interviews, proposing "both batch and stream" for different parts of the system is almost always stronger than choosing one exclusively. It shows the interviewer that you understand each model's strengths and can compose them appropriately. The Lambda and Kappa architectures covered later in this module formalize these hybrid patterns.

Common Mistakes in Interviews

Mistake 1: Defaulting to stream processing for everything. Stream processing sounds more modern and impressive, and candidates sometimes propose it for problems where batch processing is simpler and sufficient. If the problem says "generate a daily summary report," streaming that data through Apache Flink is overengineering. A batch SQL query scheduled via cron is the right answer.

Mistake 2: Ignoring the cost of always-on infrastructure. Stream processors need to run continuously. That means paying for compute 24/7, even during low-traffic periods. Batch jobs spin up, process, and shut down. For workloads that are genuinely periodic, batch processing can be significantly cheaper.

Mistake 3: Treating micro-batching as "real" stream processing. Micro-batching (as in Spark Streaming) processes small batches on short intervals. It is a useful hybrid, but it is not event-at-a-time stream processing. If the problem requires sub-100ms latency per event, micro-batching with a 500ms window will not satisfy it. Know the distinction.

Mistake 4: Forgetting about late and out-of-order data. Stream processing must handle events that arrive late or out of order. Batch processing avoids this problem because it waits for all data before processing. When proposing a stream solution, mention how you will handle watermarks, late-arriving events, and allowed lateness windows. Failing to mention this is a red flag for interviewers.

Six Takeaways

Tip:

1. Batch processing treats data as a bounded, finite thing. Stream processing treats data as an unbounded, continuous flow. The processing model you choose should match how the data naturally behaves.

2. Batch wins on simplicity and correctness. When latency requirements are relaxed (hours, days), batch processing is simpler, cheaper, and easier to debug. Do not overcomplicate a problem that a nightly cron job solves.

3. Stream wins on freshness and reactivity. When the value of processing decays with time (fraud detection, real-time alerts, live dashboards), stream processing is the right model despite its added complexity.

4. The spectrum is real. Micro-batching, windowed aggregation, and CDC occupy the space between pure batch and pure stream. Know these hybrid approaches and when each fits.

5. Re-processability is a design decision. Batch systems get it for free. Stream systems need a durable event log (like Kafka) to enable replay and correction.

6. Most production systems use both. Proposing a hybrid - stream for operational, batch for analytical - is almost always the strongest interview answer.

Exercises

Exercise: Warm-up: You are designing a system that computes the average session duration for a website. Sessions last anywhere from 30 seconds to 2 hours. The analytics team needs this metric updated once per hour. Which processing model fits, and why?

Design challenge: A ride-sharing company needs two things: (1) real-time surge pricing that adjusts every 30 seconds based on supply-demand ratio in each city zone, and (2) a daily report showing total rides, average fare, and driver utilization for each zone. Design the processing architecture for both needs. Where do batch and stream processing each fit?

Deep dive: Your team runs a nightly batch job that rebuilds the search index from scratch - processing 50 million product records in 4 hours. The business now wants search results to reflect new product listings within 5 minutes of listing creation. How would you evolve the architecture? What stays batch, what becomes stream, and how do the two interact?

Resources

• Martin Kleppmann, Designing Data-Intensive Applications, Chapter 10: Batch Processing and Chapter 11: Stream Processing - https://dataintensive.net/
• Tyler Akidau, "The world beyond batch: Streaming 101" - https://www.oreilly.com/radar/the-world-beyond-batch-streaming-101/
• Jay Kreps, "Questioning the Lambda Architecture" - https://www.oreilly.com/radar/questioning-the-lambda-architecture/
• Google Cloud, "Batch vs Streaming Data Processing" - https://cloud.google.com/architecture/batch-and-streaming-processing

Back to System Design Fundamentals