Water-cooled servers gaining steam

Vinson says the "jury's still out" on whether the complexity of having water inside the server is worth it. Air cooling inside servers is easy and less risky, he says, and customers can still gain 30 per cent energy reductions by using the HP water-cooled rack.

The US National Center for Supercomputing Applications is eagerly anticipating the arrival of Blue Waters, the 95,000-square-foot petascale computing facility on the University of Illinois campus. The NCSA, which provides computing resources to scientific engineers and industrial users, uses a 9,600-core machine that's known as "Abe" and is based on Dell blade servers. It requires three floors: the bottom floor for air handling units, the second floor for servers, and the third to handle return air flow, Pennington says.

With the 200,000-core water-cooled system, there will be a mechanical room under the server floor, with the third floor left over for office space. The data center will connect to the building's plumbing infrastructure, and from there to a large chilled water plant maintained by the university.

"We spent a significant amount of time working with people on campus and with companies, understanding how to make a water cooling room efficient," Pennington says. "I wouldn't say it's simpler [than air cooling]. It's just a different set of engineering challenges."

Pennington expects to use servers based on IBM's in-development Power7 chip, successor to the current Power6 microprocessor for high-end Unix servers, which IBM unveiled one year ago. Water cooling will be used inside the servers, Pennington notes, but the NCSA project involves considerably more work to optimize the efficiency of water cooling. Including staff, the machine room and computers, Blue Waters will cost US$208 million.

"We're providing water to the racks. IBM is doing all the other plumbing within the racks," he says.

The NCSA isn't committed to using only IBM servers. If other suitable water-cooled machines come along, the organization will buy them, Pennington says. What has been clear to Pennington for several years is that water cooling is the only viable technology that can provide the kind of power density the NCSA seeks in the foreseeable future.

After expanding the current machine room eight years ago with dense air-cooled systems, "we looked at what was coming in the next decade," Pennington says. "It was clear to us that water cooling was going to have to be a significant technology for us to think about."