Re: Snort with an expert system

[Martin Roesch <roesch () sourcefire com>]

Inline...

On Thu, Jun 25, 2009 at 5:12 PM, Richard Bejtlich<taosecurity () gmail com> wrote:
> On Thu, Jun 25, 2009 at 2:55 PM, Greg Shipley<gshipley () neohapsis com> wrote:
> > I respect the spirited and intelligent conversation here, but at the
> > risk of sounding like a) an old guy that's been following this stuff
> > for too long and b) a complete jerk:
> >
> > 1. IDS vendor / IDS software engineer / uber-geek view: "it's not
> >   technically a false-positive because if signature/ rule /
> >   pattern-match/ neugent/ whatever fired on x and it was programmed
> >   to identify q but you have to factor in y, and z, and..."
> >
> >   <bang head here -----> X
> >
> > 2. Infosec operational person trying to do his job: "Was I attacked
> >   and was the attack successful?  Yes or NO will suffice, thank you."
> >
> > I submit that for the vast majority of consumers of IDS technology we
> > really only give a crap about #2.  So if the device can give us a
> > reasonably accurate answers to #2 we are happy.  And if it can't we
> > are unhappy.
> >
> > I think the fact we've been discussing these topics for close to
> > twenty years now suggests that we aren't happy, but maybe I'm too old
> > and being a jerk.  :)
> >
> > My .01,
> >
> > -Greg
>
> Hi everyone,
>
> This is a cool debate.  I submit that it is technically impossible to
> build anything that will not 100% avoid "#2" false positives.  I'm a
> #1 guy myself; the only real "false positive" is the system telling
> you it saw something, when that something actually never happened,
> e.g., "IDS: I saw ICMP!  User: There was no ICMP; your engine isn't
> working properly."

I think the #2 case is about improving the signal to noise ratio.  I
had a group in the office a couple weeks ago who were getting 1M+
events a day from their legacy IDS deployment and that had rendered
the system effectively useless because they had no tools to assess the
impact of the detects against their deployed infrastructure.  If you
look at the Verizon report you can pretty clearly see that raw
uncontextualized detection data serves virtually no purpose in the
vast majority of deployments.  If you really want to build a useful
IDS you have to figure out how to perform that front line
contextualization in a way that's both correct and useful.  You'll
still get false positives but if you've removed 99% of the noise first
you'll have a useful detection capability anyway.

> For any case you develop that you think is absolutely, positively,
> without a doubt an "intrusion" that you could identify with an IDS, I
> can probably develop a case where that activity could turn out to be
> legitimate, and therefore, in the eyes of the organization, a "false
> positive."

This is always true but lining up what's being detected vs what an
organization can actually be vulnerable to is always going to be
useful.

> I think the "IDS" has been misnamed from the beginning.  (Blame Mr.
> Anderson?)  It should have been Attack Indication System or something
> similar.  After all "If you can detect it, why can't you prevent it?"
> Now it's really time to "bang head here."  :)

Oh man, don't get me started...


-- 
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Security for the Real World - http://www.sourcefire.com
Snort: Open Source IDP - http://www.snort.org

-----------------------------------------------------------------
Securing Your Online Data Transfer with SSL.
A guide to understanding SSL certificates, how they operate and their application. By making use of an SSL certificate 
on your web server, you can securely collect sensitive information online, and increase business by giving your 
customers confidence that their transactions are safe.
http://www.dinclinx.com/Redirect.aspx?36;5001;25;1371;0;1;946;9a80e04e1a17f194