Free lesson · Operating Systems

User Level Vs Kernel Level Threads

Easy11 min readoperating-systemsosconcurrency

Who Manages the Threads?

In the previous topics, you learned what threads are and why they matter. But there is a question we have not yet answered: who actually creates, schedules, and manages the threads? The answer to this question turns out to have major consequences for performance, and it comes up directly in interviews.

There are two places where threads can be managed - in the OS kernel, or in a library running in user space (inside your application). Each approach has different tradeoffs, and modern systems often combine both.

Kernel-Level Threads (KLTs)

A kernel-level thread is a thread that the OS kernel knows about and manages directly. When you create a kernel thread (using pthread_create() on Linux or CreateThread() on Windows), the OS creates a new schedulable entity. The kernel's scheduler can see this thread. The kernel can run this thread on any available CPU core. If this thread blocks on I/O, the kernel can switch to another thread.

This is what most people mean when they say "thread" without any qualifier. On Linux, kernel threads are created using the clone() system call with the CLONE_VM flag. Each kernel thread has a corresponding task_struct in the kernel - the same data structure used for processes. The only difference is that threads created with CLONE_VM share their address space with the parent.

Advantages of kernel threads. The kernel scheduler can distribute threads across multiple CPU cores, giving you true parallelism. If one thread blocks on I/O (like a disk read or a network call), the kernel can immediately run another thread from the same process - the blocking of one thread does not prevent other threads from running. The kernel has full visibility into all threads, which means it can make good scheduling decisions based on each thread's behavior, priority, and CPU usage.

Disadvantages of kernel threads. Creating and destroying kernel threads is relatively expensive because it requires a system call (a switch from user mode to kernel mode). Context switching between kernel threads, while cheaper than switching between processes, still involves saving and restoring registers and potentially involves the kernel scheduler's decision-making logic. For applications that need thousands or millions of concurrent execution units, the per-thread overhead of kernel threads becomes prohibitive - each thread needs its own kernel stack (typically 8-16 KB of kernel memory) in addition to its user-space stack.

User-Level Threads (ULTs)

A user-level thread is a thread that the OS kernel does not know about. The threads are managed entirely by a thread library running inside the application, in user space. The library creates, schedules, and switches between threads without any system calls. From the kernel's perspective, the application is a single process (or a single kernel thread) - the kernel has no idea that the application is running multiple threads internally.

The user-level thread library maintains its own thread table, similar to the kernel's process table but much lighter-weight. It performs context switches between user threads by saving and restoring the program counter, registers, and stack pointer - entirely in user space, without entering the kernel.

Advantages of user-level threads. Thread creation and context switching are extremely fast because no system calls are involved. Creating a user-level thread is just a function call that allocates a stack and adds an entry to the library's thread table - it can be done in nanoseconds rather than microseconds. Context switching is similarly fast - just save and restore registers in user space. This makes user-level threads ideal for applications that need very large numbers of concurrent execution units.

User-level threads are also more flexible. The thread library can implement any scheduling algorithm it wants - it is not limited to the kernel's scheduler. It can implement priority scheduling, cooperative scheduling, or domain-specific scheduling optimized for the application's workload.

Disadvantages of user-level threads. The critical problem is that the kernel does not know about user-level threads. This creates two serious issues.

First, blocking system calls block the entire process. If one user-level thread calls read() and blocks waiting for disk data, the kernel sees the entire process as blocked. Every user-level thread in the process is stuck, even though only one of them is waiting for I/O. The other threads could do useful work, but the kernel does not know they exist and cannot schedule them.

Second, no true parallelism on multiple cores. Since the kernel sees only one schedulable entity (the process or kernel thread that all user-level threads share), it can only run on one CPU core at a time. Even if the machine has 8 cores, all user-level threads within a single kernel thread are limited to one core. You get concurrency (multiple threads taking turns) but not parallelism (multiple threads running simultaneously).

Cooperative vs Preemptive Scheduling in User-Level Threads

User-level thread libraries typically use cooperative scheduling - a thread runs until it voluntarily yields control (by calling a yield function, performing I/O, or completing its work). This is different from kernel threads, which use preemptive scheduling - the kernel can interrupt a running thread at any time to give the CPU to another thread.

Cooperative scheduling is simpler to implement and avoids some concurrency problems (you know exactly when a thread gives up control), but it has a critical weakness: if a thread enters an infinite loop or performs a long computation without yielding, it monopolizes the CPU and no other user-level thread can run. Preemptive scheduling does not have this problem because the kernel's timer interrupt forces threads to give up the CPU after their time slice expires.

Modern lightweight threading systems like Go's goroutines use a hybrid approach - they are cooperatively scheduled at certain points (function calls, I/O operations) but the Go runtime also inserts preemption points to prevent goroutines from running too long without yielding. This gives the efficiency of cooperative scheduling with the fairness guarantees of preemptive scheduling.

Comparison Table: Kernel-Level vs User-Level Threads

graph TD
    subgraph KLT["Kernel-Level Threads"]
        subgraph K_PROC["Process"]
            KT1["Thread 1"]
            KT2["Thread 2"]
            KT3["Thread 3"]
        end
        subgraph K_KERNEL["Kernel"]
            KS["Kernel Scheduler<br/>sees all 3 threads"]
        end
        KT1 --> KS
        KT2 --> KS
        KT3 --> KS
        KS --> C1["Core 1"]
        KS --> C2["Core 2"]
        KS --> C3["Core 3"]
    end
    
    subgraph ULT["User-Level Threads"]
        subgraph U_PROC["Process"]
            UL["User Thread Library"]
            UT1["Thread 1"]
            UT2["Thread 2"]
            UT3["Thread 3"]
            UT1 --> UL
            UT2 --> UL
            UT3 --> UL
        end
        subgraph U_KERNEL["Kernel"]
            US["Kernel Scheduler<br/>sees 1 process only"]
        end
        UL --> US
        US --> C4["Core 1 only"]
    end

This diagram contrasts kernel-level and user-level threads. On the left, with kernel-level threads, the kernel scheduler sees all three threads individually and can assign them to different CPU cores for true parallelism. On the right, with user-level threads, the kernel sees only one process. The user thread library manages three threads internally, but the kernel can only schedule the entire process on one core. No true parallelism is possible.

The Threading Models: Many-to-One, One-to-One, Many-to-Many

The relationship between user-level threads and kernel-level threads can be organized into three models. These models describe how user threads map onto kernel threads, and they have different performance characteristics.

Many-to-One Model

Many user-level threads are mapped to a single kernel thread. The thread library manages all the user threads, and the kernel sees only one schedulable entity. This is the pure user-level threading model described above.

Performance: Thread creation and switching are extremely fast. But one blocking system call blocks all threads, and there is no parallelism across cores. This model was used in early threading implementations like Green Threads in early Java versions and GNU Pth.

Interview note: If an interviewer asks about the many-to-one model, the key point is that it provides concurrency but not parallelism, and a single blocking call can stall the entire process.

One-to-One Model

Each user thread is mapped to its own kernel thread. When the application creates a new thread, a new kernel thread is created. The kernel scheduler manages all threads and can distribute them across CPU cores.

Performance: True parallelism on multi-core machines. A blocked thread does not affect other threads. But thread creation is more expensive (requires a system call), and there is a limit on the number of kernel threads the OS can support. Creating 100,000 kernel threads is impractical - the memory overhead alone (stack space per thread) would be enormous.

This is the model used by most modern operating systems. Linux's pthreads implementation (NPTL - Native POSIX Threads Library) uses one-to-one mapping. Windows threads are also one-to-one. Java threads on modern JVMs map directly to OS threads.

Many-to-Many Model

Multiple user-level threads are mapped to a smaller or equal number of kernel threads. The thread library manages user threads and multiplexes them onto a pool of kernel threads. The kernel schedules the kernel threads across CPU cores.

Performance: Combines the advantages of both models. Thread creation is fast (just create a user thread). True parallelism is possible (via the kernel threads). A blocked system call blocks only the kernel thread it is on, not all user threads. But the implementation is complex - the library needs to coordinate with the kernel to manage the mapping.

This model is conceptually used by Go's goroutine runtime and Erlang's BEAM VM. Go's runtime creates a small number of OS threads (typically one per CPU core) and multiplexes potentially millions of goroutines across them. When a goroutine blocks on I/O, the runtime moves it off the OS thread and schedules another goroutine in its place. The next topic (Topic 06) covers this in depth.

graph LR
    subgraph M1["Many-to-One"]
        U1A["User Thread 1"]
        U1B["User Thread 2"]
        U1C["User Thread 3"]
        K1["1 Kernel Thread"]
        U1A --> K1
        U1B --> K1
        U1C --> K1
    end
    
    subgraph O1["One-to-One"]
        U2A["User Thread 1"]
        U2B["User Thread 2"]
        U2C["User Thread 3"]
        K2A["Kernel Thread 1"]
        K2B["Kernel Thread 2"]
        K2C["Kernel Thread 3"]
        U2A --> K2A
        U2B --> K2B
        U2C --> K2C
    end
    
    subgraph MM["Many-to-Many"]
        U3A["User Thread 1"]
        U3B["User Thread 2"]
        U3C["User Thread 3"]
        U3D["User Thread 4"]
        K3A["Kernel Thread 1"]
        K3B["Kernel Thread 2"]
        U3A --> K3A
        U3B --> K3A
        U3C --> K3B
        U3D --> K3B
    end

This diagram shows three threading models side by side. Many-to-One maps three user threads to one kernel thread. One-to-One maps each user thread to its own kernel thread. Many-to-Many maps four user threads across two kernel threads, with the user thread library deciding which user threads run on which kernel threads.

Interview: "Explain the threading models: many-to-one, one-to-one, and many-to-many."

"Many-to-one maps multiple user threads to a single kernel thread. It is fast for creation and switching but provides no parallelism and a blocking call stalls all threads.

One-to-one maps each user thread to a kernel thread. This is what Linux and Windows use - it gives true parallelism and independent blocking, but thread creation is more expensive and there is a practical limit on thread count.

Many-to-many maps multiple user threads to a smaller pool of kernel threads. Go's runtime uses this model - millions of goroutines are multiplexed across a few OS threads. This combines fast creation with true parallelism, but the runtime is more complex."

Why This Matters for System Design

The threading model affects architectural decisions in system design. If the interviewer asks you to handle 100,000 concurrent connections, your answer depends on the threading model available:

With one-to-one (OS threads): 100,000 connections means 100,000 kernel threads. Each thread has a ~1-8 MB stack. At 1 MB per stack, that is 100 GB of stack memory alone - impractical on any reasonable server. You cannot use thread-per-connection with OS threads at this scale. Even at 10,000 threads, the context switch overhead becomes significant - the scheduler spends more time deciding which thread to run than the threads spend doing actual work.

With many-to-many (goroutines, virtual threads): 100,000 goroutines with ~4 KB stacks each is only 400 MB - easily manageable. Go's runtime handles this effortlessly because goroutines are user-level threads multiplexed across a small number of OS threads. Java's Project Loom introduced virtual threads in Java 21, bringing the same capability to the JVM - millions of lightweight threads that map onto a small pool of OS carrier threads.

With event-driven I/O (single-threaded epoll): A single thread can handle 100,000 connections using I/O multiplexing, as Node.js and Nginx demonstrate. No threads needed at all - just asynchronous callbacks or event handlers. However, CPU-intensive work still needs to be offloaded to worker threads or the event loop will block.

This is why the choice between OS threads, green threads, and event-driven I/O is one of the most important system design decisions, and why interviewers test your understanding of it. The answer depends on the workload profile (I/O-bound vs CPU-bound), the required concurrency level (hundreds vs millions), and the programming language and runtime available.

The Evolution of Threading in Real Systems

Understanding the history helps you see why modern systems use the approaches they do.

Early Unix (1990s): Only processes. No threads in the kernel. Some user-space thread libraries existed (like Sun's LWP) but they were limited by the many-to-one model.

Linux 2.6 (2003): The NPTL (Native POSIX Threads Library) introduced efficient one-to-one kernel threads. This became the standard threading model for Linux and remains so today. Creating a thread went from hundreds of microseconds to tens of microseconds.

Go (2009): Introduced goroutines - a many-to-many model with millions of lightweight threads multiplexed across OS threads. This was designed specifically for network servers that need massive concurrency.

Java 21 (2023): Project Loom introduced virtual threads - Java's answer to goroutines. Virtual threads are lightweight threads managed by the JVM runtime, multiplexed across a pool of OS threads. This allows Java applications to create millions of concurrent threads without the overhead of OS threads.

Each generation solved the limitations of the previous one. The trend is clear: toward lighter-weight concurrency primitives that allow more concurrent execution units at lower cost.

Tip: For the interview: When discussing concurrency in system design, always mention the scale implications. "I would use a thread pool of 200 OS threads" is a different statement from "I would use goroutines" or "I would use an event loop." Each implies different memory overhead, different maximum concurrency, and different programming models. The interviewer wants to see that you understand these distinctions.

Exercise: Starter: For each of the three threading models (many-to-one, one-to-one, many-to-many), give one real system or runtime that uses it. Which model does your primary programming language use?

Intermediate: Calculate the memory overhead of creating 10,000 OS threads with 1 MB stacks versus 10,000 goroutines with 4 KB stacks. What are the numbers? What does this tell you about the scalability ceiling of each approach?

Advanced: Research how Go's goroutine scheduler works internally. How does it handle a goroutine that makes a blocking system call? What is the role of the M (machine), P (processor), and G (goroutine) in Go's scheduler model? How does this implementation achieve the benefits of the many-to-many model?

Back to Operating Systems