City of power

At just 24 years of age, you might be surprised to hear that Michael Tran is building one of the world’s largest high-density datacentres. You probably wouldn’t expect it to be in Queensland either. He recently told ARN about his grand design for a Data Centre City.

Where did you get the idea for Data Centre City?

Michael Tran (MT): We decided to take a new approach to designing and building datacentres by going to this high-density model. I’ve been in the datacentre space for six years and we were running out of power and space. It’s a global problem. Being an early adopter people have been fairly critical of what we’re doing but somebody always has to take the fi rst step and we’re proud to do that in Australia. At the end of the day it’s all about what’s available on the market – you’re now seeing a single blade server that has greater capacity than most datacentres have in a rack.

Why have people been critical?

There are two factors. Firstly, a lot of people say that chip sizes are getting smaller and they’re using less power so shouldn’t we be dropping power in the datacentre. In answer to that, 80 per cent of computing power is used by all the other resources on the motherboard rather than the processor itself. Just because chips are getting more efficient doesn’t mean the whole system is. At the end of the day there’s a cycle – density increases and we put pressure on the chip manufacturers to reduce the amount of power; they do that but the people who build the machines just shove more of them in.

The second criticism has been the size of the centres we’re building – people are questioning whether Brisbane is the right place to build what is potentially the world’s densest datacentre – but you have to look at Australia as an independent country that needs to be self-sufficient with our resources. We want to encourage the big dotcoms to come to Australia and use a local facility that meets all of their requirements. They don’t want to build big datacentres here because they don’t understand the environmental conditions.

So who do you see as the customer base?

We have a couple of vertical markets that we’re targeting – medical is seen as one of the biggest users of IT within the next few years; education is also something we want to focus on because you have a lot of archival storage and you need a lot of upfront processing power.

Those things aside, we want to take in the big dotcoms and the big hosting providers. We’ve done a lot of research in the background market and are getting a lot of other datacentres asking if they can use the facilities. It’s an interesting dynamic as an early adopter because you shake up the industry.

Tell us about your background. You seem very young to be building and managing major datacentre facilities.

I went from Web-hosting to co-location in about 15 months and very quickly started to question the space we were in. We were getting random downtime, hearing excuses from datacentres about why they were down, and it was getting difficult for us to run a business. Then we started running into the problems of running out of space and needing more power. From there I took a big step towards the medical industry and, in the five years I’ve been doing the backbone of IT in medical, we’ve seen it go from a scenario where it’s great if it’s online but we can survive if it’s not to really becoming a mission-critical application. Since it has become mission-critical we’ve seen a complete redesign and I’ve been lucky enough to be involved in upgrading datacentres and even creating new ones for many health facilities. About three years ago, my business partner and I really became serious about designing a new type of datacentre.

Can you give us some idea of the scale of Data Centre City?

We are looking at five buildings with each one powered by 36 megawatts (MW) of load capacity, the floor space is being designed to 25kW per rack, which equates to roughly 6500 Watts per square metre. It’s a full tier-four rated facility with dual path redundancy. Each of the buildings will have between 320 and 640 cabinets across the fl oor space so the final Data Centre City will have about 3200 racks.

We realised that big dotcoms or government departments don’t want to be housed in the same section as somebody else and the best section you can get is your own building. Putting five buildings in one spot means we can bring in big power feeds and Internet connections but keep the buildings completely independent of each other because they have their own transformer, power and cooling stations. We also have the ability to reroute power from one of the other buildings if it goes down for any reason.

Page Break

How are you going to power all of that?

[Queensland power supplier] Energex told us 3600MW is the capacity of Queensland and we’re looking at 192MW. There’s full redundancy built into that so technically we’re only using half of that. We’re taking feeds off the grid to two substations we have on our site, which are connected to a massive generator station. We’ve had to do a lot of complex things to ensure we have the correct level of redundancy in our generators.

When the site is finished, we’ll have one of the largest fuel storage sites in Australia. Diesel fuel goes stale because bacteria get into it so datacentres have to test their fuel on a fortnightly or monthly basis. At Data Centre City we are storing millions of litres of fuel so we need to make sure we’re not throwing that away every month because it’s expensive and bad for the environment.

When will the power be turned on?

It won’t be ready until the first [calendar] quarter of next year, because the size of investment behind it means we have to get it right from the start. Once the first building of the five is open we’ll be looking to open another one every 18 months but it will take years before they’re able to deliver full-load capacity to the substations.

That gives us time to react to the market because we’ve sat down with the IT vendors and some of them are coming out with 96kW to the rack; some of the biggest air conditioners today are 96kW so we need to be flexible in that floor space.

Have you sold the space in the first building yet?

No, it’s more of a co-location facility for customers to come in and get a feel for what this high-density monster we’re building looks like. We fully recognise that Data Centre City is a large project but there’s a need for more similar projects – some companies are looking to build their own so we want them to be able to come in and see that it can be done.

Page Break

How does the building cost compare to a traditional datacentre?

That depends how dense you want to get. The cost of building a high-density datacentre with 1000sqm isn’t very different to building a low-density datacentre with 10,000sqm but it gives you excess building capacity you can use down the track and I think that’s how people have to look at it. Using cooling systems meant we didn’t have to deploy hundreds of air conditioners around the rooms and removing them from the floor space was also a big winner.

Major server vendors tell us blades are the future and yet sales are still relatively small. Why do you think that is and do you think they will become the norm?

People have nowhere to put them. Datacentres are at limits, and consolidation has been able to help that, but they [blades] take computing power away from the datacentre and replace it with twice as much. They haven’t freed up more energy or cooling capacity and blades aren’t great for all environments. I’m tempted to say blades will become the corporate norm.

Environmental concerns around IT have grown significantly in recent times. Can a huge datacentre really be ‘green’?

We’ve really had to adapt and be seen as environmentally aware because you’ve got noise from generators and condensers, the wastage of water from evaporative cooling, diesel generators that are using masses of fuel, switchboards and UPS systems that lose power every time they convert or invert it. Selecting the right systems to get the best environmental outcome is certainly something we focused on every step of the way – we’ve gone for a no-noise approach, we’re self-reliant for water, we looked at how systems are deployed in the floor space to make sure we have the load as close to the transformers as we can, and the heat is exchanged at the closest point to the servers.

Datacentres today are running the economy and there’s no doubt if they were shut off overnight in Sydney, Brisbane or Melbourne we’d see a massive impact on the stock markets. These facilities underlie the entire infrastructure of the country.

What impact do you think the introduction of carbon reporting will have on the IT industry?

If you have to report something you are more aware so it will help people monitor more closely and see if energy saving strategies actually worked. Everybody who has to do the initial metering will probably complain but I think there are a lot of long-term gains. We’re seeing a lot of corporate social responsibility and a lot of large corporations are already doing something about it because they want to be seen as ethical companies.