nanog mailing list archives
Re: GNi/365main Above.net peering problems?
From: Rusty Hodge <rusty () hodge com>
Date: Sat, 30 Aug 2008 13:55:28 -0700
Received this update for GNi:
At this time we believe we have found a Cisco day 0 network vulnerability.We have 20+ routers in our core network - 6 of the 20 have the identical routeprocessor and IOS version. These 6 have been affected in 3 separategeographical locations in the past several days. The network issues range from simple SNMP failure to loss of BGP or OSPF communications. These are creating black holes intermittently across our network. We are experiencing asmuch as 20% of networks not available at this time.We have been coordinating with Cisco for the past two days and have deployed patches to address the problem. Overall, we have determined that the fastest, surest path to restoring 100% network normality is to replace these 6 routers.We will continue this process to replace these 6 routers.Two of the six affected routers have already been replaced and clients havebeen moved. We are now replacing the remaining 4 affected routers.We know that this week has been very painful and frustrating for you. We will provide a detailed and open RFO Reason For Outage post mortem document as soonas we have the replaced the affected routers.Also, I will be updating all customers who have opened tickets at least every two hours from this time to ensure that all customers have the latest status information. If you would prefer a phone call status report, please note iton the ticket and I will also give you a call to answer any additionalquestions you may have. In addition, you may call the NOC at your convenienceat (415) 979-9786. Thank you for your patience.
Current thread:
- GNi/365main Above.net peering problems? Rusty Hodge (Aug 30)
- Re: GNi/365main Above.net peering problems? Wil Schultz (Aug 30)
- Re: GNi/365main Above.net peering problems? Rusty Hodge (Aug 30)