In our Clear Choice Test of Microsoft's recently released 64-bit edition of Windows Server 2003, we found that when you employ optional, kernel-mode processing features, the operating system flies. When you don't, it runs a bit slower than other 64-bit server operating systems we've tested recently.
These Windows 2003 Server x64 kernel options let certain processes run at the kernel code level - in our test case SSL certificate processing, caching and session handling. When you combine these options with mandated 64-bit hardware drivers and the vast amount of memory that a 64-bit processor can address, you can get some of the best performance we've seen on Intel/AMD architectures.
When we used kernel SSL processing, the number of sustained users climbed by 90 per cent over 32-bit Windows Server 2003 processing. When compared with other 64-bit operating systems (Red Hat Enterprise Linux 4.0 [RHEL 4.0] Advance Server and Solaris 10), Windows Server 2003 x64 has a 15 per cent to 20 per cent performance advantage.
Without the kernel processing options, Windows Server 2003 x64 performed slightly under par with competitive 64-bit operating systems in our testing.
The downside to these performance gains is incompatibility issues in terms of the hardware Windows Server 2003 x64 can run on.
The two generic AMD64 whitebox systems we tested were incompatible with Windows Server 2003 x64. One wouldn't start the kernel or boot through a kernel load. The other had constant crashes after installation that seemed to be related to motherboard memory timing and additional SCSI hardware driver issues.
Two systems provided by Microsoft OEM partners - Polywell and HP - had no operational issues. Our primary test server was HP's four-way Opteron DL585 server. HP was the only hardware vendor with a full array of hardware drivers posted at Microsoft's Web site when the 64-bit operating system was released in April. This obviously limits hardware choice: something we didn't experience with the 64-bit editions of Solaris 10, SuSE SLES 9 or RHEL 4.0.
Old DOS and early 16-bit executables (games, WordPerfect 5.1, and Lotus 123 Version 4) didn't work at all or worked initially but then halted abruptly. Microsoft employs a 32-bit emulator called WOW64 that is automatically invoked to run 32-bit applications. We typically saw equal or slightly better performance of these 32-bit applications on Windows Server 2003 x64 vs. 32-bit Windows Server 2003.
Interpreted code, such as an old Visual Basic application we'd written long ago, worked very well on this 64-bit engine. And we could find no difference in execution time of a Perl script running on Microsoft's Internet Information Server Web service in the 32- or 64-bit Windows environments.
We developed an SSL Transaction script using Spirent Communications' Web Avalanche to gauge the number of sustained SSL transactions over a 10-minute build cycle.
The particular test ramps up the number of discrete user sessions, and then sustains sessions until the number of sessions dropped reaches 1 per cent. Generating SSL sessions is CPU-intensive, and managing multiple numbers of sessions presents a good gauge of how many balls the server can keep in the air before it drops one.
We tested this script against Windows Server 2003 (both 32- and 64-bit versions) and compared these numbers against the 64-bit 2.6.7 kernel in RHEL 4.0 and Solaris 10 64-bit Edition. Both systems were running Apache 2.0.3 Web service with OpenSSL. We used default settings in all cases, except when we employed the kernel-mode SSL processing on Windows Server 2003 x64, as noted.
We took two sets of Windows Server 2003 x64 measurements: one reflecting the default kernel settings, and the other reflecting the aforementioned toggle that allows SSL to be processed by the kernel.
The difference between the results were startling, and proves the benefit of this simple setting. When we ran these tests on the four-way HP DL585 server, the operating system could sustain 288,471 sessions over a 10-minute period when the SSL sessions were handled at the kernel level. The Windows Server 2003 x64 native-kernel SSL session load was fast (207,202 sessions), but not as fast as RHEL 4.0 (251,024 sessions).
We also used two prior tests for comparison - the number of maximum open TCP sessions, which measures how many can be sustained, and the number of TCP sessions per second each could support, to gauge how fast the system can ramp them up.
In the maximum TCP test, Windows Server 2003 x64 bested RHEL 4.0 but fell behind Solaris 10. In the TCP transaction per second measurements, Microsoft beat Sun, but fell behind Red Hat.
How we did it
We tested hardware-platform compatibility by testing the 64-bit Windows Server 2003 Enterprise x64 Edition on four platforms. We had no issues running the 64-bit code on our primary test platform, an HP DL585 server with four AMD Opteron 2.4-GHz CPUs and 12GB of dynamic RAM (DRAM). Nor did we have issues with the 64-bit code on our Polywell 2200S server with its dual AMD Opteron 2.8GHz processors and 4GB of DRAM. However, we had serious compatibility issues when we tried to run the code on a whitebox system with one AMD Opteron 2.8GHz processor and 1GB of DRAM, and on an Asus K8N motherboard-based server with one AMD Opteron 2.0GHz CPU and 1GB of DRAM. The official performance tests of Windows Server 2003 x64 were conducted on the HP DL585. We conducted tests in both its native mode and with its SSL-enhancements activated.
No other optimisations were used. The Windows server was both an Active Directory Domain Controller (and therefore DNS server), and a Certificate Authority. It was running Internet Information Server Version 6. A single, anonymous Web user was configured for all Web tests where applicable.
We used the same HP DL585 server to test Solaris 10, Red Hat Advanced Server 4 and SuSE Linux Enterprise Server 9; all with default settings, and Apache 2.0.3 with OpenSSL (for SSL certificate services). DNS was deployed, but LDAP, SAMBA and SQUID proxy were not used. We tested performance with two Spirent Web Avalanche systems running in parallel.
The SSL test builds a set of user SSL sessions, via HTTPS reads from the server under test, until the number of concurrent users reaches a saturation point (generating 1 per cent errors), which we term Maximum Concurrent Sessions. This test severely exercises the CPUs in the system and requires management of the user session throughout the duration of the tests through "keep-alive" connection status "tickles" from the Web Avalanche boxes.
Henderson is principal researcher for ExtremeLabs in Indianapolis. He can be reached at email@example.com.