RE: Fiber cut in SF area

[Peter Beckman <beckman () angryox com>]

Though I think networked environmental monitoring has its merits, it's
clear the technology is unproven in monitoring fiber tunnels, and my
inexperience in running and managing such tunnels makes this thread
bordering on off-topic.

I'm happy to continue conversations via email, but this will be my last
on-list reply regarding the topic I started.

On Mon, 13 Apr 2009, Crist Clark wrote:

> But would alarms prevent any, or what proportion, of these incidents?

 It's hard to say without researching.  Sometimes such research shows
 amazing results that shock people in the industry.  Hospitals were shocked
 to see surgical mistakes reduced by 80+% after implementing a checklist
 that both doctors and nurses had to go through prior to starting the
 procedure, and having the patient also go over and approve what was to be
 done.  The stories you hear of people who are getting amputated writing
 "this leg" and "X X X NOT THIS LEG" before surgery is a result of these
 studies and checklists.  RFID-tagged surgical components and gauze pads
 are another tech tool being used after such research.

 You'd think a checklist wouldn't really help, but in reality it made
 industry changing and life-saving differences.

 While active alarms and monitoring of fiber tunnels would do the same, but
 without research, nobody can say for sure how effective or ineffective
 such a system would be.

> From what we know of this specific one, would an alarm have stopped the
> perpetrator(s)? It would have bought the NOC five, ten minutes tops
> before they got the alarm on the circuit. And in practice would a manhole
> alarm translate to a call to Homeland Security to have the SEALs descend
> the site pronto, a police unit to roll by when it has the time, or is it
> going to be an AT&T truck rolling by between calls? I'm guessing number
> two or three, probably three. So what would it get them in this case. If
> it doesn't deter these guys, who does it deter?

 It's not there as a deterrent.  It's there to allow a NOC to know that
 something is going on in a tunnel where potentially critical
 infrastructure resides.  Maybe it doesn't prevent the malicious cut, but
 combined with video surveilence, it could identify the cutters.  Audio
 recording devices could record voices.

 I assume large networks have large 24/7 crews.  Get a truck to roll (once
 you sufficiently trust the system) or get a contractor who resides nearby
 to check out the area.  When the alarm goes off, you go check it.  If you
 welded the manholes shut, and there are no scheduled maintenance windows
 for that area, you can be pretty damn sure something untoward is going on,
 or it'll be a company truck roll that didn't follow procedure.

> And what are the costs of false alarms? What will the ratio of "real"
> alarms to false ones be? Maybe lower-stakes vandals take to popping the
> edge of manhole covers as a little prank.

 Weld 'em shut.  Use one of those special screws that you can only unscrew
 with the right equipment (worked wonders for the tire industry with the
 "lock nut").  It won't stop anyone determined, but 13 year olds with M80s
 will move on.  If you get a certain location that continues to get false
 alarms due to vandals, put in a highpowered webcam to monitor the
 location.  Use ZoneMinder to monitor and record motion.  Make sure the
 camera does nighttime well.  Then when you have an alarm, check the video.

> Or that one that triggers whenever a truck tire hits it right.

 I would envision that though every device would report the same data with
 the same sensitivity, false alarms could be mitigated through filters for
 a given location.  Tunnels near train tracks would be filtered differently
 than tunnels in the middle of a field under high power lines.

> Or the whole line of them that go off whenever the temperature drops
> below freezing.

 The device would go through a lot of environmental testing, so that its
 upper and lower operating limits could be known.  Hardened where
 necessary.

> Or, what I am absolutely sure will happen, miscommunication between
> repair crews and the NOC about which ones are being moved or field crews
> opening them without warning the NOC (or even intra-NOC communication).
> Will they be a boy who cried wolf?

 Maybe.  Maybe the whole idea is way too far fetched.  Maybe my impression
 of the state of affairs when it comes to fiber tunnels is really not that
 big of a deal, and that outages due to physical access (humans, backhoes,
 floods) don't make up a significant portion of outages, and this is not a
 problem that fiber companies want to solve.

 Clearly there are a lot of problems that this sort of monitoring could
 face.  Given sufficient time to mature, I think cheap, repeatable
 monitoring devices networked together can be a valuable asset, rather than
 yet another annoying alarm NOC folk and maintenance crews grow to hate and
 simply not be effective.

---------------------------------------------------------------------------
Peter Beckman                                                  Internet Guy
beckman () angryox com                                 http://www.angryox.com/
---------------------------------------------------------------------------