Those of you who read this site via RSS probably wouldn’t have noticed, but this site had some unexpected downtime this last weekend. My webhost, the always wonderful Webfaction, uses The Planet for at least some of their datacenter services, however there was a nasty hardware failure at the Huston datacenter:
[May 31st] at approximately 5:45 p.m. CDT, a transformer in one of The Planet’s Huston datacentres caught fire, requiring them to take down all of the generators on site on the instructions of the fire department. This is one of six datacentres used by WebFaction. All servers hosted at that datacentre are currently offline.
According to this Slashdot article, the outage apparently affected approximately 9,000 servers. Because of the nature of the fire, and the directives from the fire department, all the redundant power sources were made meaningless, which is kind of a crappy situation. Luckily none of the servers were actually damaged, so when the box that my site is stored on was powered up at about 6:00am this morning, all my data and services came back online with no issues.
I’m not sure if this is a situation that suggests that Webfaction should use a different datacenter, or if this is an example that goes to show that no matter how well you set up contingency plans you are always vulnerable, but I tend to think the latter. Either way, I’m impressed with how responsive Webfaction was during this event, and I’ll continue to use them as my web host.
The important thing is that the site is online and I’m back. :-D