Virtualisation’s big push to fame was arguably kick-started by VMware’s Workstation product, which allowed individual users to run a bunch of OSes, versions or instances (similar to multiple application windows) instead of having a one-at-a-time multi-boot environment. In many companies, virtualisation arrived with developers fi rst using the technology quietly to do testing and development, then introducing the virtualisation tools to IT higher-ups.
While today, computer virtualisation fuels many production environments, such as servers, desktop infrastructures, and as a provisioning tool, it is also used by a still-growing number of software developers. For starters, they use virtualisation tools to provide a range of target environments for development and testing (like different operating systems, OS versions and browsers), and also to provision/reprovision confi guration instances quickly and easily. Here’s a look at how and why some of today’s developers are using virtualisation and what their quibbles are with the technology as it stands.
Provisioning multiple test environments
Mark Friedman, a senior software architect, works in Microsoft’s Developer division, where upwards of 3000 people create Visual Studio and the .NET Framework. He works mainly on the performance tools with Microsoft’s Visual Studio Team System.
“About two-thirds of the people in my division are in development and testing – and most of these developers and testers are using system virtualisation [via Microsoft’s Hyper-V technology] as one of their key productivity tools,” Friedman said, who is also a board director of The Computer Measurement Group.
One key advantage is virtualisation’s ability to set apart an unstable environment, which is something any developer expects in early phases of application design. As the Microsoft tools are developed, Friedman said, testing early versions may destabilise a developer’s entire computing environment.
“That’s the nature of the beast,” he said. “Almost anything except the simplest desktop application can crash the system. I often tell my developers that if they aren’t crashing the system regularly, they are not trying hard enough. We appreciate virtualisation technologies, because they save time and let our developers spend more of their time on the challenging stuff, not the mundane and extremely timeconsuming aspects of prepping test environments.”
Like other virtualisation tools, Microsoft Hyper-V lets users ‘snapshot’ the system at a ‘last known good version’. “We create a rollback that allows us to restore the system to that previous good state within minutes,” Friedman said. “The alternative is having to re-image the computer or re-build the environment, which can take hours. This is a tremendous timesaver.”
Friedman’s group also makes extensive use of virtualisation and virtual machine images is in quality assurance (QA) testing. “Once you get past unit testing by the developers, we like to talk about the test matrix – what versions of the OS does it run on, for example,” he said. “Then we need to test against both the 32-bit or 64-bit versions of each. Using Virtual Machine (VM) images is the fastest, cheapest, most flexible way to maintain a variety of testing environments. In our QA labs, we are doing this all the time; we do a lot of automated testing, so we stockpile these images as VMs, and point our test suite at them. It’s invaluable. We save so much time you wouldn’t believe it.”
The ability to snapshot, and restore a working image within minutes, rather than wait hours for a rebuilt system, is particularly important. “You’re often looking at bugs where you have to spend a day or two to set-up the environment just to reproduce a bug, so the time you spend having to rebuild again and again – that’s a time waster,” Friedman said. “That is loss of productivity.”
For example, when a bug causes a system crash, the developer has to work backward to identify the point where the system crashed. “People often initially set breakpoints too far into the process and the system crashes again. Since a developer can easily be crashing the system several times a day, being able to restore quickly and then re-run the debugging session while the problem is still fresh in their minds is a great benefit,” Friedman said.
Easy to use
Virtualisation is a great boon to iterative testing, developers claim. “We have lots of test systems for use by our developers – each of which needs its own x86 control system. Rather than get a lot of PCs, even whatever’s cheapest, we’re using virtualisation to create VMs which then ‘attach’ to the external cluster boards,” senior software engineer at SiCortex, which builds and sells a range of energyeffi cient computers, Adam Moskowitz, said.
SiCortex’s development department uses the free version of VMware Server on a somewhat-beefy rack-mounted server (equipped with two or four dual-core x86, 4GB RAM, 300GB hard drive) which can run 4-8 control system VMs, Moskowitz said.
“Because there’s very little load on the control system, this works just fi ne, at lower capital expenditure. Over half of our roughly 25 engineers use these virtualised systems.”
A primary benefi t of virtualising the control systems, Moskowitz said, is ease of testing. “Want an experimental system? Clone a standard VM and away you go,” he said. The toolset is also valuable because of its ease of confi guration; developers can build a VM with the latest software once, then clone it as many times as needed.
Mike Brescia, who works for a company making real-time environmental data recording, and retrieval, said, “In addition to offering its system as a virtual appliance, our company’s developers use virtualisation to put up different test environments, for example Windows and Linux, on fewer hardware boxes.”
Cloning a clean computer system for testing purposes is much easier than copying a disk image and using Clonezilla, Brescia said.
The 15 Web developers at design and marketing fi rm, Eli Kirk, have to cater to a number of Web browsers and operating systems, its senior web developer, Connor Boyack, said. He uses Parallels version 4 on Mac OS X Leopard, and runs VMs of Windows XP, OpenSUSE 11 and Windows 7 Beta.
“Creating functional and aesthetically identical sites requires a great deal of browser testing, which makes virtualisation a must,” Boyack said.
“Virtualisation allows me to use multiple operating systems and multiple browsers all collaboratively and seamlessly (well, when Windows XP is playing nice).”
There’s a huge advantage in the ability to test a dozen different scenarios simultaneously to ensure a consistent user experience, a developer at Smart- Logic Solutions, Nick Gauthier, agreed. “Another advantage is being able to boot up a specifi c version and browser when a client has a complaint. We no longer have to say ‘Well, I don’t run that browser, I’m on Linux’; now we can say, ‘One moment please; ah, I see the problem and I’m fi xing it now’.”
“We use virtualisation to test our software on the platforms and applications we support (for QA), and to replicate customer environments when there are customer issues for support,” CTO at 3X Systems, Eric Floehr, a startup that developed and sells a remote backup appliance, said. “Virtualisation allows us to quickly bring up and tear down environments, and allows us to do so with a minimum amount of physical hardware.” The ability to test against a large number of platforms and environments with a minimum of capital outlay is especially important in a small startup company. “It greatly improves the quality and reliability of our product.”
The mobile equation Tamoggemon Software focuses on programs for handheld computers, originally for Palm OS and now also Symbian OS Series 60 and PocketPC. “I am using virtualisation a lot for various reasons,” developer, Tam Hanna, said. “My main reason is the prevalence of Cygwin/Eclipse based development systems [Eclipse for J2ME, PODS from Palm and Carbide from Nokia]. Installing them all onto one machine, well, causes all of them to break. So I need to use virtualisation in order to be able to do my job. Furthermore, I like to use VMs to keep my confi guration settings the same across all my machines.” At MarkMail, a free service for searching public mailing list archives, “our team is using virtualisation in a number of ways” developer, John D. Mitchell, said. “The key is that virtualising each and every service in the system gives us a lot of fl exibility.” MarkMail is using OpenVz so the images in the developer sandboxes are identical to the deployment environment. Using virtualisation reduces the amount of hardware they need to buy/lease and manage, according to Mitchell. “In total, we’re running about a 3 to 1 ratio of virtual instances to hardware nodes.”
The limits of virtualisation
There are, developers acknowledged, part of the process where virtualisation can’t help. “One area where virtualisation doesn’t help is in stress testing our product, which is a network-based backup appliance,” 3X’s Floehr noted. “While we could test 10 simultaneous clients virtually, it isn’t a true stress test because the 10 clients are only going through, say, four physical CPUs and two NICs, which are constraints that would not exist if they were 10 physical clients in the real world.”
“Virtualisation does add some complexity,” Mark- Mail’s Mitchell said. “It always adds some latency. But, even worse, you can’t rely on VMs to give a clear view of the performance characteristics as you change the underlying execution environment, such as going from development and testing into production.”
Finally, Mitchell said that the “current generation of desktop virtualisers still have too many annoying quirks, bugs and missing features”– at least with regard to their use by developers for software development.
“For load testing or scaling, you have to run on the raw hardware, the physical machine,” Microsoft’s Mark Friedman claimed. “Performance characteristics, unless you expect to deploy on a VM, are very different. And the same applies to the state of the art for performance measurements; you can’t trust the [physical machines] you get on a VM currently.” Although, Freidman expected this to be fixed over time.
The para-virtualisation approach used by the virtualisation industry can change some underlying aspects of the OS, notably the drivers, he added.
“So if you have a dependency on specifi c drivers or hardware environment, that will be impacted. For example, there’s a virtual NIC, and there’s a specifi c driver. The driver is ‘virtualisation-aware’. You have to understand that if you need testing against a native set of drivers, you won’t see that within a VM. So this impacts people doing driver development, but not web developers or the like.”
And, Friedman pointed out, “if you have a multitiered application”, which requires several machines, consolidating these to shared hardware means “you get a functional but not a performance test”.
Despite those quibbles, virtualisation already gives developers real benefi ts. “Virtualisation makes our developers more productive once they learn to work with the technology,” Microsoft’s Friedman said. “It allows them to spend more of their time in coding, testing and debugging. It eliminates a lot of time and delay spent in set-up and preparation, which isn’t where we want them to be spending the bulk of their time.”