VMware edges out Microsoft in virtualization performance test

How we tested virtualization products

We tested VMware's ESX 3.51 Virtual Infrastructure Edition and Microsoft's Hyper-V 1.0. For the latter we also used the Microsoft RTM release of the Linux Interconnection Kit where noted.

Our primary testing platform was an HP DL580G5 server which was connected to a switched Gigabit Ethernet network. This server was equipped with 32GB of dynamic RAM with a 1300MHz front-side bus running DDR3 memory, two Gigabit Ethernet ports, and an internal HP Smart Array P400 host bus adapter that ran a RAID 5 configuration over five drives, and a 'hot spare'. Four CPU 'sockets' were used; each containing a 2.93GHz Intel VT-capable CPU with four cores each, for a total of 16 2.93GHz 64-bit cores.

For guest operating systems running on top of the hypervisors under test, we used a combination of Windows 2008 Enterprise Edition and Novell's SUSE Linux Enterprise Server/SLES Version 10 SP2 operating system instances. We used Microsoft's LinuxIC RTM code in conjunction with SLES 10.2 when hosted as a VM on Microsoft's Hyper-V.

We used an SMP kernel for uniformity, limiting the kernel's ability to use multiple vCPUs through each test run as the results are reported.

We used Windows 2008 64-bit and SLES 10.2 x64-bit editions for all native and guest installs. We installed Windows and SLES guests using the same memory and vCPU settings. We used an SMP kernel for uniformity in all tests. Where we tested for one vCPU performance in our native vs. hypervised tests, we pulled three socketed processors from our HP DL 580G5 server. For all other testes, we made all other server cores available to the hypervisor, and then reallocated vCPUs as described in our tests, to each VM guest.

We tested performance using two benchmarks: one for business application performance called SPECjbb2005; and, one for disk IO performance, Intel's IOMeter.

We tested SPECjbb2005 on each native or virtual machine guest using an operating system memory size imposition of 2048MB, booting natively on the HP's local Smart Disk Array. We used a memory allocation process to give each instance the same amount of allocated user memory space.

We used IOMeter in pre-compiled binary form to measure disk channel/disk subsystem (an internal HP Smart Array). To tax the disk channel, we used a tougher than real-world matrix of disk channel reads/writes. We used a 30 percent read, 70 percent write mixture to exercise the drive subsystem.

All tests were run three times, each iteration compared with the others to ensure the results were within a 3 percent standard window. All results were actually much closer than the standard window allows.