nanog mailing list archives

Re: Operate until failure


From: Andy Walden <andy () tigerteam net>
Date: Mon, 8 Jan 2001 09:27:07 -0600 (CST)


On Mon, 8 Jan 2001, Shawn McMahon wrote:

We have a generator, but we have had two amazing coincidences cause it
to fail.  The first time, the generator was fine, but the switch didn't
switch.  The person who was signing off (erroneously) that he was checking
that switch monthly lost his job shortly before we stopped using his
company entirely.  We discovered the problem when the batteries reached
the point where it was supposed to cut over, and the entire data center
went dark.  That was a very, very bad day.

I recall that ominous depressing feeling sitting by myself in a dark data
center at 3 in the morning, with nothing but the lights of my equipment,
listening to the rectifiers beeping and as the batteries went dead,
hearing machines drop off one by one until it was completely dark...

andy




Current thread: