How Virtual Machines Work: From Hypervisor to Hardware

Virtual machines form the foundation of modern infrastructure, yet many engineers operate them without deeply understanding how they actually work. In this post, we’ll explore the complete stack—from hypervisor design to CPU virtualization to memory systems—to understand what’s really happening when you boot a VM.

The Hypervisor: The Core Abstraction

At the heart of any VM lies the hypervisor, a piece of software that sits between hardware and guest operating systems. The hypervisor’s job is to safely divide physical hardware resources among multiple independent VMs, each believing it has exclusive access to CPU, memory, and I/O devices.

There are two main hypervisor types:

Type 1 (Bare Metal) Hypervisors

Type 1 hypervisors run directly on hardware, making them the most efficient. VMware ESXi, KVM (when run on Linux), Hyper-V, and Nutanix AHV are all Type 1 hypervisors.

Physical Hardware
    ↓
Type 1 Hypervisor
    ↓
Guest VMs

The hypervisor handles all direct hardware access, scheduling, and resource management. This gives it complete control and visibility into the system.

Type 2 (Hosted) Hypervisors

Type 2 hypervisors run on top of a host operating system. VirtualBox and VMware Workstation are examples.

Physical Hardware
    ↓
Host Operating System (Linux, Windows, macOS)
    ↓
Type 2 Hypervisor
    ↓
Guest VMs

Type 2 hypervisors are simpler to use but less efficient since they must go through the host OS with each privileged operation.

CPU Virtualization: Making the Processor Safe to Share

The most challenging aspect of virtualization is making the CPU safe to share. Modern processors cannot actually divide their execution context—only one OS can run at a time. The hypervisor must rapidly switch between VMs, creating the illusion that each has exclusive access.

Hardware Virtualization Extensions

Modern CPUs provide hardware extensions for virtualization:

• Intel VT-x — Available on most modern Intel CPUs
• AMD-V — The AMD equivalent on Ryzen and EPYC processors

These extensions add new CPU instruction modes and capabilities that allow hypervisors to run guest code more efficiently.

Ring Model and Privilege Levels

Traditional x86-64 CPUs have four privilege levels (rings 0-3):

• Ring 0 — Kernel level, unrestricted hardware access
• Rings 1-2 — Reserved (rarely used)
• Ring 3 — User level, restricted access

Guest operating systems expect to run at Ring 0, but we can’t let them—they’d directly access hardware and crash the system. VT-x solves this with VMX (Virtual Machine Extension) root and non-root modes:

• VMX Root — Hypervisor executes here
• VMX Non-Root — Guest VM executes here, but privileged instructions cause exits

Guest Kernel (Ring 0 in VMX Non-Root)
    ↓ [Privileged instruction encountered]
    ↓ [VM Exit triggered]
    ↓
Hypervisor (VMX Root)
    ↓ [Emulate operation or handle appropriately]
    ↓
Guest Kernel (resumes)

VM Exits and Performance Implications

When a guest VM executes a privileged instruction or accesses protected resources, a VM exit occurs. The CPU switches to hypervisor context, which analyzes the situation and decides what to do.

Common causes of VM exits include:

• Privileged instruction execution
• I/O operation attempts
• Memory page faults
• Timer interrupts
• External interrupts

Each VM exit has overhead—the hypervisor must inspect the instruction, make a decision, and resume the guest. Modern hypervisors optimize this by:

1. Early exit detection — Catch problematic instructions before they execute
2. Fast path handling — Quickly service common exits (no context switches)
3. Batching — Handle multiple operations together when possible

Memory Virtualization: Isolation and Efficiency

Guest VMs believe they have exclusive physical memory, but the hypervisor must carefully manage memory to isolate VMs and optimize resource usage.

Two-Level Memory Translation

Memory addressing in virtualized systems involves two translation layers:

1. Guest Virtual → Guest Physical — Guest OS controls this via its page tables
2. Guest Physical → Host Physical — Hypervisor controls this via shadow page tables or EPT/NPT

Guest App
    ↓
Guest Virtual Address (GVA)
    ↓ [Guest Page Table]
    ↓
Guest Physical Address (GPA)
    ↓ [EPT/NPT - Extended/Nested Page Tables]
    ↓
Host Physical Address (HPA)
    ↓
Physical RAM

This dual translation provides complete isolation—guest VMs can never directly access another VM’s memory.

Extended Page Tables (EPT) and Nested Page Tables (NPT)

• EPT (Intel) and NPT (AMD) speed up memory translation by allowing the CPU to perform the GPA→HPA translation in hardware rather than forcing hypervisor intervention

Without EPT/NPT, every guest page table modification would trigger a VM exit, crippling performance. EPT/NPT provides orders of magnitude improvement.

Memory Overcommitment and Ballooning

Hypervisors often allocate more VM memory than physical memory exists—a practice called memory overcommitment.

When memory pressure occurs, the hypervisor must reclaim memory from guests. It uses several techniques:

Memory Ballooning

The hypervisor inflates a “balloon” driver inside the guest, which allocates memory. This causes the guest OS to page out its own memory, freeing physical pages for the hypervisor to use:

1. Hypervisor tells balloon driver: "Allocate 4GB"
2. Guest OS pages out memory to make room
3. Hypervisor reclaims those physical pages
4. Hypervisor can now assign them to other VMs

This is elegant because the guest OS makes intelligent paging decisions rather than the hypervisor blindly reclaiming pages.

Page Sharing

Multiple VMs often have identical memory pages (library code, common data structures, etc.). Some hypervisors use transparent page sharing—the same physical page is mapped into multiple VMs’ address spaces.

This saves significant memory in environments with many similar VMs.

I/O Virtualization: Devices and Interrupts

VMs need access to I/O devices—storage, networking, USB, etc. The hypervisor must virtualize these safely while maintaining performance.

Device Emulation

The hypervisor can emulate devices in software. When a guest accesses an I/O port or memory-mapped I/O region, the hypervisor intercepts it and simulates the device behavior.

For example, a guest might try to read from a virtual network card:

Guest OS
    ↓
Guest Device Driver
    ↓ [Reads from port 0x3F8]
    ↓ [VM Exit triggered]
    ↓
Hypervisor
    ↓ [Looks up which physical NIC this guest maps to]
    ↓ [Returns simulated network data]
    ↓
Guest OS [Receives data, thinks it came from a real NIC]

The problem with pure device emulation is that it’s slow—each I/O operation triggers intercepts and hypervisor intervention.

Paravirtualization

Paravirtualization eliminates the pretense that guests have real devices. Instead, guests explicitly use hypervisor-specific I/O mechanisms.

For example, VIRTIO (used in KVM and other hypervisors) provides:

• VIRTIO Devices — Standardized virtual device interfaces
• Shared Memory Rings — High-performance communication between guest and hypervisor

Guest App
    ↓
Guest VIRTIO Driver
    ↓ [Places operation in shared ring buffer]
    ↓ [No VM exit for most operations!]
    ↓
Hypervisor VIRTIO Backend
    ↓ [Performs actual I/O to physical device]

This dramatically reduces overhead by batching I/O operations and eliminating frequent VM exits.

Direct Device Assignment (PCI Passthrough)

For maximum performance (though reduced flexibility), the hypervisor can give a VM exclusive access to a physical PCI device:

VM
    ↓ [Direct access to physical NIC via PCI]
    ↓ [No hypervisor intervention for most operations]

This allows near-native performance but prevents live migration and sharing of the device.

Scheduling: Time-Slicing CPUs

The hypervisor must fairly divide physical CPU cores among VMs. This is similar to OS process scheduling but at a different level.

VCPU Model

Each VM gets virtual CPUs (vCPUs), which the hypervisor maps to physical CPU resources. If you have 64 physical cores and create 4 VMs with 16 vCPUs each, the hypervisor must time-slice:

Physical Core 0 Timeline:
├─ VM1 vCPU0  [time slice]
├─ VM2 vCPU0  [time slice]
├─ VM3 vCPU0  [time slice]
├─ VM4 vCPU0  [time slice]
├─ VM1 vCPU1  [time slice]
└─ (repeat)

CPU Affinity and NUMA Awareness

On large systems with NUMA (Non-Uniform Memory Access) architecture, the scheduler tries to:

• Pin vCPUs to physical cores — Reduces cache misses
• Keep vCPUs on the same NUMA node as their memory — Minimizes memory latency
• Respect hardware topology — Schedule related vCPUs together when possible

Real-World Example: Booting a VM

Let’s trace through what happens when you boot a VM:

1. Initialization — Hypervisor allocates vCPUs, memory pages, virtual devices
2. VM Entry — Hypervisor loads guest registers and executes vmlaunch instruction
3. Guest Bootloader Runs — Guest thinks it’s running at Ring 0 on real hardware
4. First Privileged Operation — Guest reads from a privileged register
5. VM Exit — CPU traps, hypervisor checks what guest was trying to do
6. Emulation — Hypervisor provides a sensible response
7. Resumption — Guest continues, unaware a VM exit occurred
8. Repeated Exits — Guest OS initialization triggers many more exits (device discovery, memory mapping, etc.)
9. Host OS Boots — Once the guest OS is running, exits become less frequent
10. Steady State — Running applications mostly don’t trigger exits; scheduling and I/O are primary concerns

Putting It Together

VM technology creates a remarkable abstraction: safe, isolated execution environments on shared hardware. This requires:

• CPU virtualization to safely trap and emulate privileged operations
• Memory virtualization to isolate guest memory and enable intelligent resource sharing
• I/O virtualization to multiplex hardware devices
• Scheduling to fairly divide CPU time

Each layer adds some overhead, but modern hardware extensions (VT-x, EPT, IOMMU) keep this overhead small in most workloads.

Understanding these mechanisms helps explain VM behavior, troubleshoot performance issues, and design better infrastructure. The hypervisor isn’t magic—it’s elegant systems engineering, using hardware capabilities to solve the fundamental challenges of safe resource multiplexing.