Free lesson · Concurrency Interview Mastery

The Producer-Consumer Pattern

Hard10 min readconcurrencymultithreadinginterview-prep

The thread pool from the last lesson was, underneath, an instance of this pattern - and so are message queues, logging systems, streaming pipelines, and request handlers across nearly every concurrent system you'll build. The producer-consumer pattern decouples threads that generate work from threads that process it, connecting them through a shared buffer so that each side runs at its own pace. Producers drop items into the buffer; consumers pull them out; neither needs to wait for the other to keep up, because the buffer absorbs the difference in their rates. It's one of the most important and most frequently asked concurrency patterns, and it's built directly from the two patterns before it: signaling (producers and consumers wake each other) and the bounded buffer that provides the backpressure of Module 6. Understanding producer-consumer - especially the bounded buffer with its two blocking conditions - is core concurrency knowledge.

What the pattern is

In the producer-consumer pattern, one or more producer threads generate items and place them into a shared buffer (a queue), while one or more consumer threads remove items from the buffer and process them. The buffer sits between them as a hand-off point, and crucially, it decouples the two roles: producers and consumers don't call each other or run in lockstep - they communicate only through the buffer, each operating independently. A producer can be bursty while a consumer is steady; there can be three producers and one consumer, or one of each; the buffer absorbs all of this.

The decoupling is the whole value. Without a buffer, a producer would have to hand each item directly to a consumer and wait for it to be processed before producing the next - tightly coupling their rates and serializing them. The buffer breaks that coupling: the producer deposits an item and immediately moves on to produce the next, while a consumer picks up items whenever it's ready. This lets producers and consumers run concurrently and at independent speeds, which is what makes the pattern a parallelism enabler, not just a hand-off mechanism. The buffer also smooths bursts: a producer that emits items in a sudden spike doesn't overwhelm a steadier consumer, because the buffer holds the surge and the consumer drains it at its own pace - turning a bursty input into a smooth one, as long as the buffer is sized to absorb typical bursts.

The bounded buffer and its two blocking conditions

The buffer almost always has a fixed capacity (a bounded buffer / bounded blocking queue), and that bound creates two conditions the pattern must handle:

Producers block when the buffer is full. If producers outrun consumers, the buffer fills; a producer trying to add to a full buffer must wait until a consumer removes something and frees space. This is what provides backpressure - a fast producer is forced to slow to the consumers' rate rather than piling up unbounded work.
Consumers block when the buffer is empty. If consumers outrun producers, the buffer empties; a consumer trying to remove from an empty buffer must wait until a producer adds something.

These two blocking conditions - wait-for-space and wait-for-item - are the heart of the bounded-buffer problem, and handling them correctly with signaling is the implementation.

flowchart LR
    P1[Producer 1] --> B
    P2[Producer 2] --> B
    B[("bounded buffer<br/>(capacity N)")] --> C1[Consumer 1]
    B --> C2[Consumer 2]
    note["Producer BLOCKS if buffer FULL (backpressure).<br/>Consumer BLOCKS if buffer EMPTY.<br/>Buffer decouples their rates."]

This diagram shows the producer-consumer pattern. On the left, multiple producer threads add items to a shared bounded buffer of fixed capacity N. On the right, multiple consumer threads remove items from the buffer and process them. The buffer sits in the middle, decoupling the two sides so each runs at its own pace. Two blocking conditions govern it: a producer blocks if the buffer is full, waiting for a consumer to free space, which provides backpressure that slows a fast producer to the consumers' rate; and a consumer blocks if the buffer is empty, waiting for a producer to add an item. The buffer absorbs differences in their rates, letting producers and consumers run concurrently and independently.

Implementation with condition variables

The canonical implementation uses a lock and two condition variables - one for each blocking condition - applying the signaling discipline from earlier:

lock = Lock()
notFull  = Condition(lock)   // producers wait here for SPACE
notEmpty = Condition(lock)   // consumers wait here for ITEMS
buffer = Queue(capacity = N)

function put(item):           // producer
    lock()
    while buffer.isFull():        # while-loop predicate
        notFull.wait()           # wait for space
    buffer.add(item)
    notEmpty.signal()            # an item is now available -> wake a CONSUMER
    unlock()

function take():              // consumer
    lock()
    while buffer.isEmpty():       # while-loop predicate
        notEmpty.wait()          # wait for an item
    item = buffer.remove()
    notFull.signal()             # space is now available -> wake a PRODUCER
    unlock()
    return item

The structure is symmetric: a producer waits on notFull until there's space, adds the item, and signals notEmpty to wake a waiting consumer; a consumer waits on notEmpty until there's an item, removes it, and signals notFull to wake a waiting producer. Using two condition variables (rather than one) means each signal targets exactly the group that cares - adding an item wakes a consumer, freeing space wakes a producer - avoiding the wrong-waiter problem from the signaling exercise. The while-loop predicates handle spurious and stolen wakeups (Module 6), which matters especially with multiple producers or consumers, where a thread can wake to find another already took the item or space.

Implementation with semaphores

The classic alternative uses semaphores, which is elegant because semaphores count:

emptySlots = Semaphore(N)    // counts free space (starts at capacity)
fullSlots  = Semaphore(0)    // counts items (starts empty)
mutex      = Mutex()         // protects the buffer structure

function put(item):           // producer
    emptySlots.acquire()         # wait for / take a free slot (blocks if none)
    mutex.lock(); buffer.add(item); mutex.unlock()
    fullSlots.release()          # signal: one more item available

function take():              // consumer
    fullSlots.acquire()          # wait for / take an item (blocks if none)
    mutex.lock(); item = buffer.remove(); mutex.unlock()
    emptySlots.release()         # signal: one more free slot
    return item

Here emptySlots counts available space (starts at N) and fullSlots counts available items (starts at 0). A producer acquires an empty slot (blocking if the buffer is full), adds the item under the mutex, and releases a full slot. A consumer acquires a full slot (blocking if empty), removes under the mutex, and releases an empty slot. The semaphores handle the counting and the blocking - and because semaphores remember their count (unlike condition variables), there's no lost-wakeup gap. The mutex separately protects the buffer's internal structure from concurrent modification. This semaphore solution is famous for its elegance and is the textbook answer to the bounded-buffer problem.

Backpressure and why the buffer must be bounded

The bounded buffer's full-condition is what gives the pattern backpressure - and it's why the buffer must be bounded, not unbounded. With an unbounded buffer, a producer that outpaces consumers would never block; it would keep adding items, and the buffer would grow without limit until it exhausts memory (the queue-leakage / OOM failure of Module 6). A bounded buffer forces a fast producer to wait when the buffer fills, naturally throttling it to the consumers' processing rate. This is backpressure: the system pushes back on the source of work when it can't keep up, rather than silently accumulating an unbounded backlog and collapsing.

Important: The producer-consumer pattern decouples work-generating threads from work-processing threads via a shared bounded buffer, letting them run concurrently at independent rates. Two blocking conditions define it: producers block when the buffer is full (providing backpressure), consumers block when it's empty. Implement it with a lock + two condition variables (notFull for producers, notEmpty for consumers - so each signal reaches the right group) and while-loop predicates, or with two semaphores (emptySlots counting space, fullSlots counting items) plus a mutex for the buffer. The buffer must be bounded - an unbounded buffer removes backpressure, so a fast producer grows it until OOM. This pattern is the basis of thread pools, message queues, and pipelines. Knowing both implementations and the bounded-for-backpressure rationale is the complete answer.

Scaling to multiple producers and consumers

A key strength of the pattern is that it generalizes cleanly from one-of-each to many producers and many consumers - the same buffer, the same blocking conditions, more threads on each side. This is where the pattern becomes a parallelism engine rather than just a hand-off:

Multiple consumers let you scale processing: if one consumer can't keep up with the producer, add more, and they pull from the same buffer in parallel. This is exactly the thread pool - many worker (consumer) threads draining a shared task queue fed by submitters (producers).
Multiple producers let you aggregate work from many sources into one stream: several threads generating events all feed one buffer, consumed by a processing tier. A logging system where every application thread is a producer of log lines, drained by one or a few writer threads, is this shape.
Many-to-many combines both: a pool of producers and a pool of consumers communicating through one buffer, each side scaled independently to match its workload.

The correctness requirements intensify with more threads, which is why the while-loop discipline matters so much here. With a single consumer, a stolen wakeup can't happen (there's no one to steal from); with multiple consumers, a single item can unblock several, and only the while loop keeps the losers from acting on a buffer another consumer already emptied (Exercise Part 3). Likewise the buffer's internal mutation must be mutex-protected because multiple producers (or consumers) can now contend on the structure simultaneously. So scaling the pattern is "add threads to either side," but it requires the full discipline - two condition variables (or counting semaphores), while-loop predicates, and a mutex on the structure - to stay correct under the concurrency it enables. The reward is a pattern that scales to high throughput by independently sizing the producer and consumer tiers to their respective rates, with the buffer absorbing the mismatch.

Real-world incarnations

Producer-consumer is so foundational that it appears, often unnamed, across systems software - and recognizing the pattern beneath each makes them easier to reason about:

Thread pools (last lesson): submitters produce tasks, workers consume them, through a task queue. A thread pool is a multi-producer, multi-consumer bounded buffer.
Message queues (Kafka, RabbitMQ, SQS): producers publish messages, consumers subscribe and process - producer-consumer at distributed scale, with the queue as a durable, networked buffer. The same blocking/backpressure concepts apply (consumer lag, queue depth limits).
Logging systems: application threads produce log entries into a buffer; a dedicated writer thread consumes them and flushes to disk or network, so the application threads aren't blocked on slow I/O. The buffer decouples fast producers from slow output.
Streaming and data pipelines: each stage produces records consumed by the next, connected by buffers - which is the pipeline pattern (a later lesson), essentially chained producer-consumer stages.
Request handling: an acceptor thread produces incoming connections/requests into a queue; a worker pool consumes and handles them - the standard server architecture.

The unifying observation is that any time work is generated in one place and processed in another at a different rate, producer-consumer with a bounded buffer is the structure that connects them - decoupling the rates, enabling parallelism on each side, and providing backpressure when the consumers can't keep up. Seeing this one pattern beneath thread pools, message queues, logging, pipelines, and servers is exactly the kind of recognition that lets you understand and design concurrent systems rather than treating each as a separate special case. It's the most pervasively useful pattern in the module.

Note: Producer-consumer is worth recognizing as the concurrency answer to a universal problem: whenever two parts of a system operate at different speeds, you need something between them to absorb the mismatch, or the faster part is forced to wait on the slower one. In single-threaded code you might solve this with a buffer or batch; in concurrent code, the bounded buffer with its two blocking conditions is the canonical solution, because it not only absorbs the mismatch but coordinates the threads safely. The deeper principle - decouple components that run at different rates through a buffer - recurs at every scale, from a CPU's instruction pipeline to a hardware FIFO to a distributed message queue spanning data centers. The threading version you've built here is one instance of a pattern that pervades all of computing. Carrying that recognition means that when you see a rate mismatch between concurrent components, "put a bounded buffer between them" is the reflex, and you already know how to make that buffer correct.

The interview angle

Interview: Among the most common concurrency questions. (1) "What's producer-consumer?" - Producers add items to a shared buffer, consumers remove and process them; the bounded buffer decouples them so they run at independent rates. (2) "What are the blocking conditions?" - Producers block when the buffer is full (backpressure); consumers block when it's empty. (3) "How do you implement it?" - Lock + two condition variables (notFull/notEmpty, while-loop predicates), or two semaphores (emptySlots/fullSlots) + a mutex. (4) "Why two condition variables, not one?" - So each signal wakes the right group (adding an item wakes a consumer, freeing space wakes a producer); one shared CV with signal() can wake the wrong waiter. (5) "Why bounded?" - An unbounded buffer has no backpressure, so a fast producer grows it until OOM. Being able to write both the condition-variable and semaphore versions, and explain the bounded-buffer backpressure, is the strong answer.

Quick reference

The pattern that connects work-generators to work-processors through a buffer that absorbs their rate difference - the basis of thread pools, message queues, logging, and pipelines. The two things to get right are the two blocking conditions (full→producer waits, empty→consumer waits) and signaling the correct group, and the buffer must be bounded so a fast producer meets backpressure instead of exhausting memory. Two implementations are worth knowing cold - the lock-plus-two-condition-variables version and the two-counting-semaphores-plus-mutex version - since interviewers often ask you to write and contrast both:

PRODUCER-CONSUMER: producers add items to a shared BOUNDED BUFFER, consumers remove + process
  buffer DECOUPLES them -> they run concurrently at INDEPENDENT rates

TWO BLOCKING CONDITIONS:
  producer BLOCKS when buffer FULL  -> BACKPRESSURE (throttles a fast producer)
  consumer BLOCKS when buffer EMPTY -> waits for an item

CONDITION-VARIABLE IMPL (lock + TWO condition vars):
  put:  lock; while full: notFull.wait(); add; notEmpty.signal(); unlock
  take: lock; while empty: notEmpty.wait(); remove; notFull.signal(); unlock
  TWO CVs so each signal wakes the right group; WHILE loops (spurious/stolen wakeups)

SEMAPHORE IMPL (semaphores count + remember):
  emptySlots=Sem(N), fullSlots=Sem(0), mutex
  put:  emptySlots.acquire(); lock;add;unlock; fullSlots.release()
  take: fullSlots.acquire(); lock;remove;unlock; emptySlots.release()

BUFFER MUST BE BOUNDED: unbounded = no backpressure -> fast producer grows it -> OOM

USED IN: thread pools, message queues, logging, streaming pipelines, request handling

Producer-consumer governs shared data that flows through a buffer from writers to readers. The next pattern governs shared data that's accessed in place by many readers and occasional writers - the reader-writer pattern, which allows concurrent reads but exclusive writes, optimizing the common read-heavy case.

Back to Concurrency Interview Mastery