US Data Center chillers not backed up by diesel generators?

Rackspace are a high-profile data centre in the US. They had a couple of outages in the last few days which have badly damaged their reputation. The main outage, according to Rackspace, happened when:

at approximately 6:30 PM CST Monday, a vehicle struck and brought down the transformer feeding power to the DFW data center. It immediately disrupted power to the entire data center and our emergency generators kicked in and operated as intended. When we transferred power to our secondary utility power system, the data center’s chilling units were cycled back up. At this time, however, the utility provider shut down power in order to allow emergency rescue teams safe access to the accident victim. This repeated cycling of the chillers resulted in increasing temperatures within the data center. As a precautionary measure we decided to take some customers’ servers offline.

When I read this, something about it didn’t seem right. I couldn’t understand why the chillers (the machines which cool the water for the aircon) would need to be power cycled. Then, an explanation showed up on the Texas Startup blog:

It turns out that in most multi-tenant commercial property in the United States, the building owner provides chilled water so that tenants can run their HVAC systems. In general, most buildings do NOT put these chillers on power with generator backup

If this is true, it is frightening!

In other words, if power fails to these buildings, the diesel generator will ensure that their aircon will be circulating air which is rapidly increasing in temperature because the water is no longer being chilled.

I’m director of Cork Internet eXchange, the first professional data centre in Ireland outside of Dublin and I can absolutely guarantee that our chillers are on power with a backup from our diesel generator. Of course they are. Why would you design it any other way?

Om Malik put it well when he said:

our Internet infrastructure, despite all the talk, is as fragile as a fine porcelain cup on the roof of a car zipping across a pot-holed goat track

4 thoughts on “US Data Center chillers not backed up by diesel generators?”

  1. Tom,

    What’s the timescale for the open of CIX? Your tone and the buildup over the last series of months has that reassuring feeling of standing on a great big rock. I’m excited to see how it’ll go when you launch.

    Paul

  2. Thanks Paul. To be honest we’ll have more of a soft launch than a grand opening.

    We have been held up considerably getting the ESB and MV contractor to come to site on the same day to throw the final switch so we will have power to the data centre floor. We now expect that to happen lte this week so we should be in a position to start taking in servers by next month at the latest.

  3. Hi Tom,

    Have you had any contact from the boys and girls in microsoft about their data centre plans in Ireland? They are on a green agenda by the sounds of it.

    The thing about cooling in Texas is that it’s damned hot down there. Diesel backup for the cooling would take quite a lot of generation. (Of course that doesn’t mean that you don’t need it, just that it’s hard to do.)

    Good luck with the soft launch!

  4. Rackspace would have to have their chillers on generators (my reading is that they do). They said themselves that it would only last but minutes before the thermal cut out would kick in if the chillers went down.

    It was something to do with the switch over from: generator -> esb -> generator (for repair crew to work on the damaged transformer) in a short space of time, not allowing the chillers/fans to get up to full speed between supply switch over.

    Thats what I picked up from the various posts. I also didnt see much information about the exact downtimes people experienced!

    -steve

Comments are closed.