Free lesson · Python Basic to Advance

Concurrency Concepts

Hard7 min readpython

Concurrency Concepts

Concurrent vs Parallel

Two related but distinct terms:

Concurrent - multiple tasks make progress over the same time window, possibly interleaving on a single CPU
Parallel - multiple tasks run literally at the same time on multiple CPUs

A single-core machine can run concurrent tasks (via time-slicing) but not parallel ones. A multi-core machine can do both.

# Concurrent - overlapping execution, may use one or multiple cores
import asyncio
async def fetch_many():
    await asyncio.gather(fetch_a(), fetch_b(), fetch_c())

# Parallel - multiple processes running simultaneously
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor() as ex:
    results = list(ex.map(compute, range(100)))

This code contrasts two concurrency patterns. The async version interleaves three I/O-bound tasks. The process pool actually distributes work across CPU cores. Concurrency is about structure; parallelism is about execution. Most concurrent code aims for parallelism when possible.

I/O-Bound vs CPU-Bound

The single most important distinction for choosing a concurrency model:

I/O-bound - spends most of its time waiting (network requests, disk reads, database queries)
CPU-bound - spends most of its time computing (math, image processing, parsing)

# I/O-bound - waiting for network
def fetch_url(url):
    response = requests.get(url)    # mostly waiting
    return response.text

# CPU-bound - actively computing
def compute_factorial(n):
    result = 1
    for i in range(2, n + 1):
        result *= i
    return result

This code shows the distinction. Fetching a URL spends almost all time waiting for the network. Computing a factorial spends all time using the CPU. The right concurrency tool depends on which side dominates: threads or async for I/O-bound, processes for CPU-bound.

The wrong concurrency tool can make code slower than the serial version.

Python's Concurrency Tools

Python provides three main concurrency models:

Model	Best for	Trade-offs
Threads (`threading`)	I/O-bound	Shares memory, but limited by the GIL for CPU work
Processes (`multiprocessing`)	CPU-bound	True parallelism, but separate memory; pickling overhead
Async (`asyncio`)	I/O-bound at scale	Single-threaded, cooperative, no GIL issue

# Threads - I/O-bound
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor() as ex:
    results = list(ex.map(fetch_url, urls))

# Processes - CPU-bound
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor() as ex:
    results = list(ex.map(compute, inputs))

# Async - I/O-bound at high concurrency
import asyncio
async def main():
    results = await asyncio.gather(*(fetch(url) for url in urls))

This code shows three concurrency tools. Thread pool for I/O. Process pool for CPU work. Async for very high I/O concurrency. The choice depends on the workload - covered in detail through this module.

The GIL - Briefly

Python's Global Interpreter Lock (GIL) ensures only one thread executes Python bytecode at a time. This means:

Multiple threads can't speed up CPU-bound Python code
Threads still help for I/O-bound code (the GIL releases during I/O waits)
For CPU parallelism, use processes (or libraries that release the GIL like numpy)

Topic 22.2 covers the GIL in depth.

Race Conditions

When multiple threads or processes access shared data, the order matters - and Python doesn't guarantee a safe order without explicit synchronization:

import threading

counter = 0

def increment():
    global counter
    for _ in range(100_000):
        counter += 1    # not atomic!

threads = [threading.Thread(target=increment) for _ in range(10)]
for t in threads: t.start()
for t in threads: t.join()

print(counter)    # not 1_000_000 - race condition

This code has a race condition. Multiple threads increment the same counter. The plus equals operation is not atomic - read, add, write. Two threads can read the same value and overwrite each other's writes. The final value is unpredictable and almost always less than expected. Topic 22.4 covers synchronization tools that fix this.

Race conditions are subtle bugs - they may not show up in testing but appear under production load.

Deadlocks

A deadlock happens when threads (or processes) wait on each other in a cycle:

import threading

lock_a = threading.Lock()
lock_b = threading.Lock()

def thread_one():
    with lock_a:
        time.sleep(0.1)
        with lock_b:    # waiting for thread_two to release
            ...

def thread_two():
    with lock_b:
        time.sleep(0.1)
        with lock_a:    # waiting for thread_one to release
            ...
# Both threads wait forever

This code shows deadlock. Thread one holds lock A and wants B. Thread two holds B and wants A. Neither can proceed. Common in concurrent code that takes multiple locks. The fix is consistent lock-acquisition order or higher-level abstractions.

The classic prevention: always acquire locks in the same order across all code paths.

Amdahl's Law

A theoretical limit on speedup from parallelism. If a fraction p of a program is parallel and 1-p is serial, with N processors the maximum speedup is:

Speedup = 1 / ((1-p) + p/N)

Speedup = 1 / ((1-p) + p/N)

This formula expresses Amdahl's law. The serial portion is the bottleneck. A program that's 90 percent parallel hits a ceiling of 10x speedup, no matter how many processors. The first lesson of optimization: identify and minimize the serial portion before adding parallelism.

In practice, most Python programs are limited by I/O, the GIL, or coordination overhead - Amdahl's bound is rarely the actual ceiling.

Coordination Overhead

Concurrency isn't free. Several costs:

Context switching - kernel time spent swapping threads
Synchronization - locks, queues, atomic operations
Memory consistency - caches, memory barriers
Serialization - pickling data for process communication

For small workloads, these costs exceed the benefit of concurrency. Serial code is often faster for small inputs.

import time

def serial_sum(values):
    return sum(values)

def threaded_sum(values, n_threads=4):
    # Overhead of thread creation often dominates for small inputs
    ...

This code hints at the overhead cost. For small inputs, the serial version is often faster than threaded - the thread creation and coordination costs exceed the actual work. Concurrency earns its place when the work itself is significant relative to overhead.

The rule of thumb: parallelize work that takes much longer than the synchronization overhead.

When Concurrency Helps

A few patterns where concurrency clearly pays off:

Many slow I/O calls - fetch 100 URLs concurrently rather than serially
CPU-bound work with multi-core - distribute heavy computation
Real-time responsiveness - keep a UI responsive while doing background work
Server handling many clients - async handles thousands of connections

# Many slow I/O calls - async helps
async def fetch_many(urls):
    return await asyncio.gather(*(fetch(url) for url in urls))

# CPU-bound - processes help
def process_images(paths):
    with ProcessPoolExecutor() as ex:
        return list(ex.map(process_one, paths))

This code shows where concurrency pays off. Many slow operations benefit from running concurrently. CPU-bound work benefits from spreading across cores. The performance gain depends on how much waiting or computing each task does relative to coordination cost.

When Concurrency Doesn't Help

Concurrency adds complexity. Avoid it when:

Work is small - overhead dominates
Tasks share heavy state - synchronization erases gains
The bottleneck is elsewhere - disk, database, network upstream
Debugging is critical - concurrent bugs are hard to reproduce

# Don't parallelize a quick computation
def slow_version():
    with ProcessPoolExecutor() as ex:
        results = list(ex.map(lambda x: x + 1, range(1000)))    # overhead dominates

# Faster serially
def fast_version():
    results = [x + 1 for x in range(1000)]

This code shows when not to concurrentize. Adding one to a thousand numbers is so fast that any concurrency overhead dwarfs the actual work. The serial version is much faster. Profile first; concurrentize only where it actually helps.

Choosing a Model

A decision framework (Topic 22.15 covers in depth):

A handful of CPU-bound tasks → ProcessPoolExecutor
Many CPU-bound tasks → multiprocessing.Pool
A handful of I/O-bound tasks → ThreadPoolExecutor or just threading.Thread
Many I/O-bound tasks (hundreds-plus) → asyncio
Mixed workload → start with asyncio and offload CPU work to processes

For most application code, the standard library has the tool you need.

Mental Model

A useful way to think about concurrency:

Threads - multiple workers sharing one workshop (memory)
Processes - multiple workshops with separate stockrooms
Async - one worker juggling many tasks, never blocking

Each model has trade-offs in safety, complexity, and performance.

Performance Notes

For Python specifically:

Thread creation: tens of microseconds
Process creation: tens of milliseconds (much slower)
Async task creation: microseconds
Lock acquire/release (uncontended): nanoseconds
Inter-process communication: milliseconds (involves serialization)

Choose the granularity to match: small tasks → async; large tasks → threads or processes.

Interview Relevance

Interview: Concurrency basics are core Python:

Concurrent vs parallel - the distinction

I/O-bound vs CPU-bound - choosing the right tool

The GIL and its implications

Race conditions and deadlocks

When concurrency helps vs hurts

The three Python concurrency models

A common interview moment: "speed up this script that downloads 100 URLs." The right answer involves asyncio or ThreadPoolExecutor - I/O-bound, threads or async help.

Another moment: "speed up this CPU-heavy computation." The right answer involves ProcessPoolExecutor - bypass the GIL.

Module 30 covers Python interview preparation.

Common Pitfalls

Warning: Threading for CPU-bound work. The GIL prevents speedup. Use processes instead.

Concurrency for small tasks. Overhead exceeds savings. Profile first.

Sharing mutable state without locks. Race conditions. Use synchronization or message passing.

Acquiring locks in different orders. Deadlocks. Pick one order and stick to it.

Assuming += is atomic. It isn't. Use threading.Lock or atomic types.

Mixing concurrency models carelessly. Threads inside async, async inside threads - each has rules.

Debugging concurrency bugs in production. Reproduce locally with high contention; use careful logging.

Try It

Exercise: Practice with concurrency concepts:

Write a small program that fetches 10 URLs serially vs concurrently with threads. Compare timings.

Try ThreadPoolExecutor on a CPU-bound function. Compare with serial execution - note the lack of speedup.

Try ProcessPoolExecutor on the same CPU-bound function. Note the speedup.

Reproduce a race condition with two threads incrementing a counter. Verify the result is inconsistent.

Design a deadlock scenario with two locks. Trigger it.

Identify an I/O-bound and a CPU-bound task in code you've written. For each, decide which concurrency model would fit.

Back to Python Basic to Advance