Virtualisation benchmark wars break out - again

Benchmark wars are back in the virtualisation world as VMware rebutted Oracle's claim that its newly-released virtualisation hypervisor is three times faster than VMware's.

Oracle released its Xen-based hypervisor, Oracle VM, on 12 November, claiming that it was "three times more efficient than existing products from other vendors" - a thinly veiled reference to market-leading vendor VMware, and a claim made repeatedly in several of its marketing materials, even in the same document.

VMware blogger Steve Wilson, who works for Sun Microsystems, disputed Oracle's claim. In his blog, copied onto VMware's site, he said:

"Despite the reports about 3x faster, what Oracle said was actually, 'Oracle consistently saw much better resource utilisation with an average of three times less overhead using Oracle VM.' In benchmark-speak, that's a lot different than 3x faster. In fact, I'm guessing that someone constructed a benchmark that shows Oracle VM running at 99 percent of native speed, while VMware ran at 97 percent of native speed - 1 percent overhead instead of 3 percent. Yes, 'three times less overhead', but ultimately trivial."

However, the spat may not be that serious. VMware points out that its hypervisor is optimised for use with large databases such as Oracle.

And on Oracle's release of Oracle VM, VMware said it welcomed Oracle's entry into the virtualisation space, but regarded Oracle as more of a partner than a competitor. "We hope this will be the first of many steps that Oracle takes towards broad enablement of virtualisation," said Parag Patel, vice president of alliances for VMware.

Wilson added: "Of course, in the long term, there will be better official benchmarks of VM performance, but the real test will be customer metrics like:

• How fast can I roll out a new application?
• How many system admins do I need to run my datacentre?
• How much energy can I save by operating my datacentre in a new way?