nanog mailing list archives
Re: Famous operational issues
From: Brian Knight via NANOG <nanog () nanog org>
Date: Thu, 18 Feb 2021 18:10:18 -0600
On 2021-02-17 13:28, John Kristoff wrote:
On Wed, 17 Feb 2021 14:07:54 -0500 John Curran <jcurrran () istaff org> wrote:I have no idea what outages were most memorable for others, but the Stanford transfer switch explosion in October 1996 resulted in a much of the Internet in the Bay Area simply not being reachable for several days.Thanks John. This reminds me of two I've not seen anyone mention yet. Both coincidentally in the Chicago area that I learned before my entry into netops full time. One was a flood: <https://en.wikipedia.org/wiki/Chicago_flood> The other, at the dawn of an earlier era:<http://telecom-digest.org/telecom-archives/TELECOM_Digest_Online/1309.html>I wouldn't necessarily put those two in the top 3, but by some standard for many they were certainly very significant and noteworthy. John
Thanks for sharing these links John. I was personally affected by the Hinsdale CO fire when I was a kid. At the time, my family lived on the southern border of Hinsdale in the adjacent town of Burr Ridge. It was weird like a power outage: you're reminded of the loss of service every time you perform the simple act of requesting service, picking up the phone or toggling a light switch. But it lasted a lot longer than any loss of power: It was six or seven weeks that, to this day, felt a lot longer.
Anytime we needed to talk to someone long-distance, we had to drive to a cousin's house to make the call. To talk to anyone local, you'd have to physically go and show up unannounced. At 11 years old, I was the bicycle messenger between our house and my great-grandmother, who lived about two blocks away. My mother and father kept the cars gassed up and extra fuel on hand in case there was an emergency.
Dad ran a home improvement business out of the house, so new business ground to a halt. Mom worked for a publishing company, so their release dates were impacted. The local grocery store's scanners wouldn't work, so they had to punch the orders into the register by hand, using the paper sticker prices on the items.
I clearly remember from the local papers that they had to special-order the replacement 5ESS at enormous cost. I saw the big brick building after the fire with the burn marks around the front door. In late May and early June, the Greyhound buses with the workers were parked around the block, power plants outside with huge cables snaking in right through the wide open front door.
When we heard that dial tone at last, everyone was happier than an iPhone with full bars. Lol
We're spoiled for choice in telecom networks these days. Also, facilities management have learned plenty of lessons since then. Like, install and maintain an FM-200 fire suppression system. But nevertheless, sometimes when I step into a colo, I think of that outage and the impact it had.
-Brian
Current thread:
- Re: Famous operational issues, (continued)
- Re: Famous operational issues Sean Donelan (Feb 16)
- Re: Famous operational issues Mark Andrews (Feb 16)
- Re: Famous operational issues scott (Feb 16)
- Re: Famous operational issues Joe (Feb 16)
- Re: Famous operational issues Rich Kulawiec (Feb 16)
- Re: Famous operational issues Miles Fidelman (Feb 17)
- Re: Famous operational issues Justin Wilson (Lists) (Feb 17)
- Re: Famous operational issues David Guo via NANOG (Feb 17)
- Re: Famous operational issues Jared Mauch (Feb 17)
- Re: Famous operational issues Justin Wilson (Lists) (Feb 17)
- Re: Famous operational issues Sean Donelan (Feb 16)
- Message not available
- Re: Famous operational issues John Kristoff (Feb 17)
- Re: Famous operational issues Brian Knight via NANOG (Feb 18)
- Re: Famous operational issues John Kristoff (Feb 17)
- Re: Famous operational issues John Curran (Feb 17)
- Re: Famous operational issues George Metz (Feb 18)
- Re: Famous operational issues Wolfgang Tremmel (Feb 19)
- Re: Famous operational issues Ben Cannon (Feb 21)
- Re: Famous operational issues Daniel Karrenberg (Feb 19)
- Re: Famous operational issues Sabri Berisha (Feb 19)
- Re: Famous operational issues Aaron C. de Bruyn via NANOG (Feb 19)
- Re: Famous operational issues Andrew Gallo (Feb 19)
- Re: Famous operational issues Warren Kumari (Feb 19)
- Re: Famous operational issues Clayton Zekelman (Feb 20)