|
A message from our Servers: |
|
|
Written by Daniel
|
|
Monday, 02 June 2008 |
Info:
From Doug Erwin:
This evening at 4:55pm CDT in our H1 data center, electrical gear shorted, creating an explosion and fire that knocked down three walls surrounding our electrical equipment room. Thankfully, no one was injured. In addition, no customer servers were damaged or lost.
We have just been allowed into the building to physically inspect the damage. Early indications are that the short was in a high-volume wire conduit. We were not allowed to activate our backup generator plan based on instructions from the fire department.
This is a significant outage, impacting approximately 9,000 servers and 7,500 customers. All members of our support team are in, and all vendors who supply us with data center equipment are on site. Our initial assessment, although early, points to being able to have some service restored by mid-afternoon on Sunday. Rest assured we are working around the clock.
We are in the process of communicating with all affected customers. we are planning to post updates every hour via our forum and in our customer portal. Our interactive voice response system is updating customers as well.
There is no impact in any of our other five data centers.
I am sorry that this accident has occurred and I apologize for the impact.
More updates from the datacentre
"""
As you know, we have vendors onsite at the H1 data center. With their help, we've created a list of equipment that will be required, and we're already dealing with those manufacturers to find the gear. Since it's Saturday night, we do have a few challenges.
We are prioritizing issues as follows:
Getting the network up at H1 is first and foremost. We're pulling components from our five other data centers - including Dallas which will be an all-night effort.
Getting power back to the data center is key, though it is too early to establish success there.
Because ServerCommand is in H1, our legacy EV1 customers are blinded about this incident. We are in the process of moving the ServerCommand servers to other Houston data centers so that we're able to loop them into communications.
We absolutely intend to live up to our SLA agreements, and we will proactively credit accounts once we understand full outage times. Right now, getting customers back online is the most critical... [comments...] |
