AlgoInsight

Free lesson · Java Basic To Advance

JVM Memory Architecture

Hard8 min readjava

Two Memory Models, Same Name

Nikhil here, opening Module 19. Before we go any further, we need to clarify something that confuses thousands of Java engineers: Java has two unrelated "memory models," and they're both important.

Memory Model 1: The Java Memory Model (JMM). Covered in Module 18, Topic 18.3. Defines what threads can observe about each other's writes to shared memory. About happens-before, visibility, ordering. Specified in JLS Chapter 17.

Memory Model 2: The JVM Memory Architecture. Covered in this module. Defines how the JVM organizes memory at runtime — heap, stack, metaspace, native memory. About allocation, lifetimes, garbage collection.

The two are unrelated. JMM is the concurrency contract. JVM Memory Architecture is the runtime layout. When senior engineers say "memory model," they could mean either — context matters.

This Module 19 is about the second one. Where objects live. How they're allocated. When they're collected. How to find leaks. How to tune. Where the heap dump is when you crash with OutOfMemoryError.

The High-Level View

When a JVM process runs, its memory is organized into several regions:

┌─────────────────────────────────────────────────────┐
│                                                     │
│  Native memory (OS-managed, outside JVM control)    │
│  ┌───────────────────────────────────────────────┐ │
│  │                                               │ │
│  │  JVM-managed memory                          │ │
│  │  ┌─────────────────────────────────────────┐ │ │
│  │  │  Heap (objects)                         │ │ │
│  │  │  - Young generation                     │ │ │
│  │  │  - Old generation                       │ │ │
│  │  └─────────────────────────────────────────┘ │ │
│  │  ┌─────────────────────────────────────────┐ │ │
│  │  │  Metaspace (class metadata)             │ │ │
│  │  └─────────────────────────────────────────┘ │ │
│  │  ┌─────────────────────────────────────────┐ │ │
│  │  │  Thread stacks (one per thread)         │ │ │
│  │  └─────────────────────────────────────────┘ │ │
│  │  ┌─────────────────────────────────────────┐ │ │
│  │  │  Code cache (compiled JIT code)         │ │ │
│  │  └─────────────────────────────────────────┘ │ │
│  └───────────────────────────────────────────────┘ │
│                                                     │
│  Direct buffers, native libraries, etc.            │
└─────────────────────────────────────────────────────┘

Each region has a different purpose, lifetime, and management strategy. Let's walk through each.

The Heap

The heap is where Java objects live. Every new allocation, every Integer.valueOf(), every record instantiation — all go on the heap.

User user = new User("alice");   // User object on heap
List<String> list = new ArrayList<>();   // ArrayList on heap, internal array on heap

The heap is:

The heap is the largest memory region in most JVM processes — typically gigabytes. Topic 19.2 covers it in depth.

The Stack

Each thread has its own stack. The stack stores:

public void method() {
    int x = 5;              // x lives on the stack
    User u = new User();    // u (the reference) lives on stack; User object on heap
    helper(x);              // new stack frame pushed
}                            // frames popped on return

The stack is:

The stack stores REFERENCES to heap objects, not the objects themselves. When you write User u = new User(), the reference u is on the stack; the User object is on the heap.

Topic 19.7 covers stack mechanics in detail.

Metaspace

Metaspace stores class metadata — the runtime representation of every class loaded by the JVM:

public class User { ... }   // Class metadata in Metaspace

When the JVM loads User.class (the first time someone references it), the class definition goes into Metaspace.

Metaspace:

Before Java 8, this region was called PermGen (Permanent Generation) and was part of the heap. The size was hardcoded by -XX:MaxPermSize. Many production OutOfMemoryError: PermGen problems led to its replacement.

Topic 19.8 covers Metaspace, class loaders, and class memory.

Thread Stacks

Each thread allocates its own stack. The JVM default is typically 512KB-1MB per thread, configurable with -Xss.

For platform threads, this is OS-allocated memory. Creating 1000 threads consumes ~1GB just in stack space.

For virtual threads (Java 21+), stacks are much smaller and managed differently. See Topic 18.14.

Code Cache

The JIT compiler (Just-In-Time) compiles frequently-executed bytecode into native machine code for performance. This compiled code lives in the code cache.

You rarely think about the code cache, but it's a real region. If you see CodeCache is full. Compiler has been disabled. in your logs, the cache is exhausted.

Native Memory

Outside the JVM-managed regions:

Native memory is outside the heap and the GC's control. Direct buffers are freed when their wrapper Java object is collected, but with timing the GC chose.

For services using NIO or native libraries, native memory can rival the heap in size.

Where Each Type of Data Lives

A concrete example. Trace through this code:

public class Order {
    private final UUID id;
    private final List<Item> items;
    
    public Order(UUID id, List<Item> items) {
        this.id = id;
        this.items = items;
    }
}

public void process(Order order) {
    String name = order.getName();   // local variable
    int count = order.getItems().size();
    
    // ...
}

When the JVM executes this:

On the stack (for the current thread):

On the heap:

In Metaspace:

The heap holds the data. The stack holds references and primitives. Metaspace holds the class definitions.

This separation is critical to understand. When we discuss garbage collection (Topic 19.3+), we're talking about reclaiming heap memory. When we discuss thread limits, we're talking about stack memory. When we discuss "too many classes," we're talking about Metaspace.

A Real Production Concern

A common bug at scale: each request creates "just a few" objects, but cumulatively the rate exceeds what the GC can handle.

In a Cloud Spanner client service:

If your code is producing garbage faster than the GC can collect, you're going to have a bad time. The fix is usually one of:

This module gives you the tools to recognize, measure, and address these patterns.

A Summary Table

Region Stores Per-thread? Garbage collected? Configurable
Heap Objects No (shared) Yes -Xmx, -Xms
Stack Frames, locals, refs Yes No (LIFO) -Xss
Metaspace Class metadata No Limited -XX:MaxMetaspaceSize
Code cache JIT-compiled code No Limited -XX:ReservedCodeCacheSize
Native Direct buffers, JNI Varies Indirectly OS-managed

What This Module Will Cover

The remaining topics:

By the end, you should be able to read a heap dump, identify a memory leak, tune the GC for your workload, and understand what happens when you call new in a hot loop.

graph TD
    A["JVM Process Memory"] --> B["JVM-managed"]
    A --> C["Native (OS-managed)"]

    B --> D["Heap<br/>(objects)"]
    B --> E["Thread Stacks<br/>(frames, locals)"]
    B --> F["Metaspace<br/>(class metadata)"]
    B --> G["Code Cache<br/>(JIT-compiled)"]

    D --> H["Young Generation"]
    D --> I["Old Generation"]

    C --> J["Direct ByteBuffers"]
    C --> K["Memory-mapped files"]
    C --> L["Native libraries (JNI)"]

    M["Two 'Memory Models'"] --> N["JMM (Module 18)<br/>happens-before, visibility"]
    M --> O["JVM Memory (Module 19)<br/>heap, stack, metaspace"]
    M --> P["Unrelated despite shared name"]

    style A fill:#7c3aed,color:#fff
    style M fill:#dc2626,color:#fff

This diagram organizes JVM memory architecture. The JVM-managed regions: the heap holds objects allocated via new and is divided into Young and Old generations for generational garbage collection; thread stacks hold method frames, local variables, and references with one stack per thread; Metaspace holds class metadata — the runtime representation of every loaded class, replacing the older PermGen in Java 8; the code cache holds JIT-compiled native code. Native memory is OS-managed, outside JVM control: direct ByteBuffers for NIO, memory-mapped files via FileChannel, native libraries loaded via JNI. Critical distinction: Java has two memory models with the same name. The Java Memory Model (JMM) covered in Module 18 defines happens-before and visibility for concurrency. The JVM memory architecture covered in this Module 19 defines runtime memory layout. They are unrelated despite the shared term — context determines which is meant.

Common Pitfalls

Warning: Confusing the JMM with JVM memory architecture. Different concepts, same name.

Thinking "more heap = better performance." Larger heaps mean longer GC pauses.

Ignoring stack size for high-thread services. 1MB × 1000 threads = 1GB just in stacks.

Forgetting Metaspace exists. Heavy reflection or dynamic class loading can fill it.

Treating direct buffers as heap. They're native memory, freed differently.

Why This Matters in Interviews

Interview: "Describe JVM memory architecture" is a standard senior question. "Where does a String literal live?" "What happens to a local variable when a method returns?" "What's the difference between heap and stack?" are all routine.

Topic 19.11 covers the Interview Lens.

Try It

Exercise: Get oriented:

  1. Run a simple Java program with -XX:+PrintGCDetails and observe the memory regions reported.
  2. Use jcmd <pid> VM.native_memory (with -XX:NativeMemoryTracking=summary enabled) to see native memory breakdown.
  3. Identify in your codebase: which classes are loaded? Use -verbose:class to see.
  4. Allocate a 1GB direct buffer. Verify it doesn't affect heap.
  5. Configure -Xss512k and run a deeply recursive program. Observe StackOverflowError.
  6. Run with explicit -Xms/-Xmx/-XX:MaxMetaspaceSize and verify the regions cap at those values.

Each exercise builds memory architecture intuition.

Back to Java Basic To Advance