Some virtualization technologies (eg. Xen and L4Linux) are hypervisor based, while others (eg. KVM) use Linux as the basis for the virtualized platform. Before answering the obvious question - Which is better, hypervisor based or Linux based? - here is a short explanation of the difference.

How things works

In a normal - not virtualized - GNU/Linux system the Linux kernel controls the hardware and all the processes running on the system.

In Linux based virtualization, virtual machines are managed in a way similar to normal processes in the system. If desired, the Linux kernel can be modified to treat the virtual machines a little different from normal processes, for example giving them a different scheduling priority or keeping the virtual machines locked in memory, but basically they can be regarded as having some similarity to processes.

Hypervisor based systems, like Xen, do not have Linux in full control of the hardware. Instead, a smaller piece of software, the hypervisor, controls certain things in the system:

• CPU time scheduling.
• Physical memory allocation and protection between virtual machines.
• Interrupt routing.
• Time keeping.
• Message passing between virtual machines.

In the case of Xen, typically one of the virtual machines takes care of the other hardware bits, like device drivers for disk, network, video card, etc. The other virtual machines communicate with that first privileged virtual machine (domain 0) to manipulate their virtual devices. A hypervisor can be seen as a form of microkernel.

There are a number of advantages and disadvantages to each approach.

Hypervisor advantages

• The system is controlled by a small piece of software, small enough to be audited for security issues.
  • Rebuttal: in practice the code that lives in domain 0 needs to be audited too, so the software is no longer small.
• The hypervisor takes up little virtual address space, which matters on 32 bit systems.
• The exact same kernel can be run as host OS (domain 0) and guest OS.
  • Rebuttal: This advantage is disappearing with coopvirt or the paravirt ops infrastructure.
• The same hypervisor can be used with multiple operating systems.
  • Eg. use Linux or FreeBSD as the host OS, with the same virtualization software.
  • Rebuttal: With VMI or a clean coopvirt interface, this could also be done without a hypervisor.
• The hypervisor underlies all kernel-based isolation safeguards with an additional level of safeguards to isolate guests from each other.

Linux based advantages

• The virtualization software can be smaller, because it shares more functionality with the Linux kernel.
  • Rebuttal: Because all guests share the same underlying linux kernel, each guest is only as isolated from the others as any linux process is from other processes.
• Deep-down OS functionality like kexec+kdump just work, without needing support in the hypervisor.
  • This includes things like CPU frequency scaling and laptop suspend/resume, which do not currently work in Xen.
• The same is true for other hardware support that would need hypervisor support, eg. pmtimer or some ACPI functionality.
• Ubiquity. Once a Linux based virtualization technology is in the upstream (kernel.org) kernel, it can be enabled very easily by Linux distributions.
• Virtual Memory. A hypervisor like Xen only manages physical memory, running under a Linux kernel allows for Virtual Memory paging and oversubscription.
• Lower context switching times: running natively under Linux with a paravirt aware guest and cr3 caching, context switching can actually exceed that of Xen PV domains.

• Lower IO overhead: As all IO under a hypervisor setup such as Xen is done by a userspace daemon in domain0, the overhead to do any paravirt IO must include the domain->domain context switch as well as the kernel->userspace context switch to the daemon to handle the IO request. If IO is instead handled by the Linux kernel natively, the only context switch is to kernel space to handle the IO request.

• Linux will probably evolve to provide many of the benefits of a hypervisor solution, while retaining its existing benefits as a virtualization platform.

So, which is better?

Which approach is better depends entirely on what your goals are. For a Linux kernel maintainer (hi Linus) it is much easier if the software is small, simple and entirely inside the kernel. Interfaces to an external hypervisor mean an extra interdependency with another project.

For end users the functionality will probably be more important than what the underlying technology is. As long as the virtualization technology in question is efficient, easily manageable and available on the system by default it will do the job. End users will be interested in the Linux virtualization technology comparison table.

In the end, chances are that whatever technology makes it into the upstream (kernel.org) kernel and fulfills all of a user's technical needs will be the one that gets used. Practical availability trumps theoretical advantages every time.