AlgoInsight

System Design

Threads, Cores, and Context Switching: How Your Code Actually Runs

You hit 'run' and your program springs to life. But what actually executes it? Follow the journey from the main function to threads racing across CPU cores - and the switching trick that keeps everything moving when there aren't enough cores to go around.

By Admin2026-07-0914 min readconcurrencythreadscpucontext-switchingparallelismsystem-design

You write some code, hit "run," and your program springs to life. But have you ever stopped to ask what actually executes it? Where does execution begin? What physically runs your instructions? And what happens when your program tries to do many things at once? The answers take you from a single function all the way down to the silicon of your CPU - and reveal one of the most important ideas in all of computing: the difference between running things truly at the same time and just appearing to. This is the journey of your code, from main to multiple cores.

The entry point: what is main?

Every program begins somewhere, and that somewhere is the main function (or method). When you run a program, the runtime doesn't start at a random line - it looks for main and starts there. main is the entry point: the very first piece of your code that the runtime invokes when your program launches.

Think of it as the front door of your program. No matter how many functions, classes, and files your program contains, execution enters through main and flows out from there. Everything your program does begins as a consequence of main running. This is true across most languages - C, C++, Java, Go, Rust, and others all designate a main as the entry point (some scripting languages have variations, but the concept of a single entry point holds). Understanding that execution has one clear starting point is the first step to understanding how code runs.

// Execution begins here - main is the entry point
public class Program {
    public static void main(String[] args) {
        System.out.println("This is the first thing that runs");
        // ... the rest of your program flows from here
    }
}

The main thread

Here's where it gets interesting. When your program enters main, the operating system doesn't just start running code in the abstract - it assigns your program a thread of execution. Specifically, it gives you a single default thread called the main thread.

A thread is a sequence of instructions that the CPU executes one after another. The main thread is the default one that runs your code sequentially - line by line, in order. When you write straightforward code with no special concurrency, everything runs on this single main thread: line 1, then line 2, then line 3, each completing before the next begins. This sequential, one-thing-at-a-time execution on the main thread is the default model of how programs run, and it's exactly what you'd intuitively expect: your code runs top to bottom, one step at a time.

So the picture so far: execution enters at main, and it runs on the main thread, sequentially. For many programs, that's the whole story. But modern programs often want to do more than one thing at once - and that's where threads, cores, and the real magic come in.

CPU and cores

To understand what happens next, you need to know what's physically running your threads: the CPU (Central Processing Unit) - the "brain" of the computer that executes instructions.

Here's the key detail: a modern CPU contains one or more cores. A core is an independent processing unit - essentially a complete engine capable of executing instructions on its own. And this is the crucial fact: each core can execute exactly one thread at any given instant. One core, one thread, at a time. A single core is like a single worker who can do one task at a time; it's fast, but it's fundamentally doing one thing in any given moment.

For a long time, CPUs had a single core, so they could truly execute only one thread at a time. But modern CPUs are multi-core - they pack multiple cores onto one chip (dual-core, quad-core, eight-core, and beyond). And that changes everything about what a program can do at once.

Parallel execution with multiple cores

If your CPU has multiple cores, something powerful becomes possible: it can execute multiple threads simultaneously - one thread per core. A quad-core CPU can genuinely run four threads at the exact same instant, each on its own core. This is true parallelism.

This is the whole point of multi-core processors, and why they made computers faster in a way that just cranking up clock speed couldn't. If your program can split its work across multiple threads, and your CPU has multiple cores, those threads run at the same time, and the work finishes faster. A task that would take one core four seconds might take four cores roughly one second, because the four cores work on it in parallel. This is what makes multi-core systems faster for multi-threaded workloads: real, simultaneous execution across cores.

graph TD
    subgraph PARALLEL["True Parallelism (4 threads, 4 cores)"]
    C1[Core 1] --> T1[Thread A]
    C2[Core 2] --> T2[Thread B]
    C3[Core 3] --> T3[Thread C]
    C4[Core 4] --> T4[Thread D]
    end
    subgraph SWITCH["Context Switching (4 threads, 1 core)"]
    CX[Core 1] --> TS[Thread A ↔ B ↔ C ↔ D<br/>rapidly switching]
    end

This diagram contrasts two situations. In true parallelism, four threads run on four cores - Thread A on Core 1, Thread B on Core 2, Thread C on Core 3, and Thread D on Core 4 - all executing simultaneously, one thread per core. In context switching, four threads must share a single core, so the core rapidly switches between Thread A, B, C, and D, running each for a small slice of time. Parallelism runs threads at the same instant; context switching interleaves them on one core.

But there's a catch, and it's the heart of the whole story: what happens when you have more threads than cores?

What happens when threads exceed cores?

Real programs often create many threads - sometimes dozens or hundreds - while your CPU has only a handful of cores. So what happens when the number of threads exceeds the available cores? A core can only run one thread at a time, so the CPU physically cannot run all those threads simultaneously. There simply aren't enough cores to go around.

Instead, the CPU does something clever: it rapidly switches between the threads, giving each one a small slice of time on a core before moving to the next. Thread A runs for a tiny moment, then the core switches to Thread B for a moment, then Thread C, and so on - cycling through all the threads, each getting brief turns. This switching happens so fast (many times per second) that it appears all the threads are running at once, even though at any single instant, each core is really only running one.

This rapid switching between threads is called context switching, and it's how a computer with, say, four cores can juggle hundreds of threads. It creates the illusion of many things happening simultaneously, by slicing time finely and sharing the cores.

Context switching and performance overhead

Context switching sounds like a free lunch - juggle unlimited threads on limited cores! - but it isn't free. Understanding its cost is what separates a surface understanding from a real one.

When the CPU switches from one thread to another, it has to save the state of the current thread (where it was in its execution, the values it was working with - everything it needs to resume later) and load the state of the next thread (restoring where that thread left off). This save-and-restore work takes time and CPU cycles - and critically, during that switching, the CPU isn't doing your program's actual work; it's doing the bookkeeping of switching. This is overhead.

For a reasonable number of threads, this overhead is small and well worth it - it lets the system stay responsive and make progress on many tasks. But here's the trap: excessive context switching can degrade performance rather than improve it. If you create far too many threads, all competing for a few cores, the CPU spends more and more of its time switching between threads and less and less time actually running them. The threads compete for CPU time, the switching overhead piles up, and the whole program can end up slower than if you'd used fewer threads. More threads is not always better - past a point, it makes things worse.

Warning: A common beginner mistake is assuming "more threads = faster." It's not true. Threads run truly in parallel only up to the number of cores; beyond that, they share cores through context switching, which adds overhead. Spawning hundreds of threads on a quad-core machine doesn't make your program four-hundred times faster - it can make it slower, as the CPU drowns in context-switching overhead. The right number of threads depends on the work and the hardware, not "as many as possible."

Concurrency vs. parallelism: the key distinction

Everything above points to one of the most important distinctions in computing - one that trips up even experienced engineers, and one that interviewers love to probe. The two ideas are parallelism and concurrency, and they are not the same thing.

The classic way to hold the distinction: parallelism is doing many things at once; concurrency is dealing with many things at once (by interleaving them). Parallelism is about simultaneous execution; concurrency is about managing multiple tasks that make progress together, whether or not they run at the same instant.

Put simply: parallelism needs multiple cores (it's literally simultaneous). Concurrency does not - a single core can be concurrent by rapidly switching between tasks. A single-core machine can run many threads concurrently (via context switching) but cannot run them in parallel (it has only one core). A multi-core machine can do both: run some threads in parallel across cores, and handle more threads than cores concurrently through switching.

This distinction matters enormously in practice. When you design a system to do many things at once, you're designing for concurrency. Whether that concurrency achieves true parallelism depends on the hardware (how many cores) and the nature of the work. Understanding that concurrency is the broader concept (making progress on many tasks) and parallelism is the specific case (running them simultaneously on multiple cores) is foundational to reasoning about performance.

When does adding threads actually help?

If more threads isn't always faster, when does threading help? The answer depends on what your threads are actually doing, and it's the practical payoff of everything above.

There are two broad kinds of work. CPU-bound work keeps the core busy computing - crunching numbers, processing data, running calculations. For CPU-bound work, threads help up to the number of cores: four cores can genuinely crunch four threads' worth of computation in parallel, but a fifth thread has no free core, so it just adds context-switching overhead without adding real compute. So for pure computation, the useful number of threads is roughly the number of cores - beyond that, you're just paying switching costs.

I/O-bound work is different - it spends most of its time waiting (for a network response, a disk read, a database query) rather than computing. Here, threads help even beyond the core count, because a thread that's waiting on I/O isn't using its core - it's idle, blocked. While one thread waits, the core can switch to another thread that has work to do. So for I/O-bound work, having many threads (more than cores) is genuinely useful: while some wait, others progress, and the core stays busy instead of sitting idle during the waits. This is why a web server handling thousands of slow network connections can benefit from many threads (or other concurrency models) even on a few cores - most threads are waiting most of the time.

The practical rule that falls out: match your concurrency to your workload. For CPU-bound work, roughly one thread per core (more just adds overhead). For I/O-bound work, more threads than cores can help, because waiting threads free the core for others. Understanding why - that CPU-bound work saturates cores while I/O-bound work leaves them idle during waits - is what lets you choose the right level of concurrency instead of guessing. It's the direct, practical consequence of knowing how threads, cores, and context switching actually interact.

Bringing it all together

Let's retrace the whole journey. You hit run, and execution enters at main, the entry point. The OS gives your program a main thread that runs your code sequentially. Your instructions execute on a CPU, which contains one or more cores, each able to run exactly one thread at an instant. If you have multiple threads and multiple cores, they run in parallel - truly simultaneously, one per core, which is what makes multi-core systems fast. But when threads exceed cores, the CPU context-switches - rapidly sharing cores among threads, creating the illusion of simultaneity (concurrency) at the cost of overhead, which becomes harmful if overdone.

And underneath it all sits the distinction that ties it together: parallelism (many things running at once, needing multiple cores) versus concurrency (many things making progress by sharing cores). That single idea - the difference between actually simultaneous and effectively simultaneous - is one of the most important concepts a software engineer can internalize.

Interview: Threads, cores, and the concurrency-versus-parallelism distinction are among the most common interview topics, and getting the distinction crisp is a strong signal. Be ready to state it clearly: parallelism is multiple threads executing simultaneously on multiple cores; concurrency is multiple threads making progress by sharing cores through context switching (which doesn't require multiple cores). Add the practical insight - more threads than cores means context switching, which has overhead, so "more threads" isn't always "faster" - and you demonstrate real understanding. Interviewers love the follow-up "what's the difference between concurrency and parallelism?" precisely because so many people conflate them; nailing it sets you apart.

The bottom line

Your code's journey from "run" to execution goes like this: it starts at main (the entry point), runs on the main thread (sequentially by default), and executes on a CPU made of one or more cores (each running one thread at a time). Multiple threads on multiple cores run in true parallelism - simultaneously - which is what makes multi-core systems fast. But when threads exceed cores, the CPU context-switches, rapidly sharing cores to create concurrency - the illusion of simultaneity - at the cost of switching overhead, which harms performance if excessive.

The idea that ties it all together: parallelism is doing many things at the same instant (needs multiple cores); concurrency is making progress on many things by interleaving them (works even on one core). Internalize that distinction, and you understand not just how your code runs, but how to reason about making it fast. We cover concurrency, threading, and performance across System Design Fundamentals.

Go deeper

More articles