Re: Network hardware IPS

["Stefano Zanero" <zanero () elet polimi it>]

> They claim a "92% reduction in false positives".

Sometimes this kind of bragging makes me wonder: these people actually think
they are speaking to clueless folks ? Or the average audience is actually
inclined to hear "92%" and then run to buy a copy of whatever they are
selling ?

False Positive rate and Detection Rate are inversely proportional, in any
detection system. It is true for radars, it is true for medical screening
systems, it is true for anything. Check out the ROC, receiver operating
curves concepts.

I cannot draw in an e-mail, but you can pick up a sheet of paper yourself.
Draw a graph: on x-axis, put FP rate. On y-axis, put DR.

Now think of a clueless, totally clueless, intrusion detection system. It
generates totally random answers. You CAN obtain a 100% detection rate with
it - if you accept a 100% false positive rate. The diagram on your chart is
a line, bisecting the quadrant. If you want a 50% detection rate, you need
to accept a 50% false positive rate, and so on.

Better intrusion detection systems would have a different graph, which
stands "above" the diagonal line. Draw it - it's just any curve you may
think of, which (hopefully !) is monotonically increasing, starting from
(0,0) and ending up in (100,100).

Do you notice something ? You _CAN_ reduce by any factor (92%, 95%,
99.9999%) the FP rate - but you WILL, always, without doubt, pay a price in
detection rate terms. You can do it for the "idiot" IDS described above, you
can do it for the best IDS you may think of: but it has always got a price!

The curve gives you a suggestion: the best "working point" is the one where
the rate of FP increase vs. DR increase is at its top. Of course,
determining it in reality is not as simple as on our simple equation ! But
this model explains clearly (even clear enough for a salesperson maybe) that
"decrease in false positive" or "increase in detection rate" mean nothing at
all, by themselves.

Stefano Zanero



---------------------------------------------------------------------------
Captus Networks IPS 4000
Intrusion Prevention and Traffic Shaping Technology to: 
 - Instantly Stop DoS/DDoS Attacks, Worms & Port Scans
 - Automatically Control P2P, IM and Spam Traffic
 - Precisely Define and Implement Network Security & Performance Policies
FREE Vulnerability Assessment Toolkit - WhitePapers - Live Demo 
http://www.securityfocus.com/sponsor/CaptusNetworks_focus-ids_000101
---------------------------------------------------------------------------