[M5Hosting] Emergency Maintenance and Initial Network/Power Event
Overview
Michael J McCafferty
mike at m5computersecurity.com
Wed Mar 18 21:00:08 PDT 2009
Dear M5Hosting Customer,
As you may be aware, your server was rebooted today due to a
momentary power loss at the data center facility (American Internet
Services) from which M5Hosting provides your service. The root cause
was a failed power system component during a scheduled generator test.
After the power was restored, M5Hosting also had some network issues
on our "PremiumNet" due to some redundant network components behaving
in an unexpected manner.
The failed power system components have been replaced and power will
be taken off generator and put back on the "street" power tonight
during an emergency maintenance window declared by the data center
facility. This maintenance window will begin at 8:30pm US/Pacific
Time. While the window has been declared to be 2hrs, the actual
cut-over will be quite short. The window will allow for testing. This
work is not expected to impact service at all.
I have pasted the most recent announcement from the data center
facility below. We will also forward any subsequent announcements from
them regarding this incident.
While the network and facility are fully functional and stable at
this time, there is some work to do return all systems back to full
redundancy and normal operating status. The maintenance discussed
below will be the first step. This will return the power system to
it's normal operating status. Our network work will take place during
a late night maintenance window to be announced later and scheduled to
mitigate any risks of an impact to service.
There will be more detailed follow up explanation of the events today.
If you are still experiencing any issues with your server or other
services which we provide for you at this time, please contact us at
877-344-4678 or submit a support ticket by email to
support(at)m5hosting.com, so we take care of your needs immediately.
Sincerely,
Mike
The most recent Data Center facility announcement from AIS:
[quote]
To Our Valued Customers,
Following this morning's facility issue at the 9725 Scranton Data
Center, AIS Facilities and Engineering teams are announcing a 2 hour
emergency maintenance window which will begin at 17:15 and which will
conclude at or before 20:15.
During this window, main power for the facility will be returned from
generator to normal operating mode. Throughout the day, engineering
and mechanical teams have taken every precaution and performed
considerable diagnostics to ensure that all systems are once again
operating at 100% effectiveness. Even following these precautions,
all maintenance windows involving critical power systems still carry
the risk of causing of a power event during the maintenance operation,
thus, we are bringing this important issue to your attention at this
time. While maintaining a cautious posture, we at AIS do not
anticipate any issues during this window.
During the maintenance window, the facility will remain fully staffed
by 18 customer service engineers, 3 different mechanical engineering
contractors, and AIS senior management personnel.
We are in the process of producing a detailed post incident report
which will be made available during business hours tomorrow, Thursday
March 19th.
A follow up communication will be sent upon completion of the
maintenance window. We apologize for any inconvenience this issue may
have caused.
AIS Support Services
[/quote]
--
************************************************************
Michael J. McCafferty
Principal, Engineer
M5 Hosting
877-344-HOST x501 Voice
http://www.m5hosting.com
************************************************************
More information about the M5Hosting
mailing list