RE: False Positives with IntruVert

[Bill Boyle <bboyle () intruvert com>]

OK.  I'll bite.  *courteous disclaimer: obviously from my email I work for a
vendor.  Also, these are my personal opinions may or may not reflect that of
my company.  That being said, the goal of this email is not to sell anyone
product but to put into perspective the CURRENT value of IPS. 

This is not the first email to come across the list saying that IPS is an
unusable product because of false positives.  What I have not seen on this
list is the statement "IPS is not an all or nothing technology."  Nor have I
seen the statement "I have recently tested and IPS solution and..."  An IPS
solution, like any solution, is not a silver bullet solution.  There isn't a
vendor out there who is claiming that their solution eliminates false
positives or is the end-all solution.  It is another tool.  

There are clearly attacks that will 100% not false positive.  My question to
the community is: Why would you NOT block this traffic on your network?  We
have all been burned with false positives and I am sure that is where the
mistrust originates.  But, a lot of technology has been put forth to more
accurately identify attacks since IDS's inception.  

What I see as necessary for IPS to be useful TODAY is not zero false
positives.  It is performance, accuracy, and granularity of policy.

PERFORMANCE:  First and foremost, the box sitting on the wire must process
the packets at wire speed and not drop ANY packets. It must also have little
downtime (same expectations as my switch and router or firewall
infrastructure).  

ACCURACY:  Exemplified in Paul Schmehl's email, a signature alone is not
always accurate enough.  Correlation must be used to enhance the accuracy.
For example, Intruvert uses Boolean logic in its signature set to provide
more accurate detection.  Example: sig#1 in URI request AND (sig2 OR sig3 in
body).

Not every signature is 100% accurate but it does not mean it is not of use.
Example: a TCP connection on port 5432.  It may be backoriface or may be a
random source port that happened to be a well know backoriface port.  It
will be a false positive but one that I would want to review.  The fact that
I got this alert does not mean my IPS is worthless, it just means that I
will not choose to automatically block this event.  Other backoriface
attacks that have a lower false positive rate will (at MY choosing-not the
vendor's).  

The ability to drop the single packet or single flow is necessary.  Broad
responses such as drop all port 80 to a specific server because  of a web
attack might leave me open to spoofing and denial of service.

In every IDS product there are a range of signatures and anomaly alerts.
Some are dead accurate, others are suspicious but indispensable.  Events
like these need to be rated in the console: This is a HIGH severity attack
but a MEDIUM chance that this is a false positive.  

GRANULARITY OF POLICY: In order for an IPS to be effective, the rules must
be allowed to be applied with precision.  Example: Only this individual Code
Red attack gets blocked if it is an inbound attack destined for IIS
machines.  The second Code Red Attack (also comprised of multiple signature
matches) will alert only for any direction.  This precision is what makes
IPS usable.  You do not want to say block ALL ATTACKS.  You could, but you
won't want to.  A usable product will permit the flexibility in any
environment.

My 2cents.

--bill 
bboyle () intruvert com


I have also responded inline to the questions below: 


-----Original Message-----
From: Cure, Samuel J [mailto:scure () kpmg com] 
Sent: Friday, March 28, 2003 12:36 PM
To: 'focus-ids () securityfocus com'
Subject: False Positives with IntruVert 

Looking for some feedback on IntruVert.  I have a client that is evaluating
IntruVert in the lab and has been getting a lot of false positives on their
network.  They are afraid to put IntruVert into the IPS mode, of actually
stopping traffic based on false positives.  Gartner Group has claimed that
everyone is moving from Detection to Prevention, but if the underlying
technology has this many flawed signatures, I do not see how anyone can
confidently use it and start blocking all attacks. 

Boyle: Please contact me offline.  I would be interested in seeing what the
false positives are.  Are they informational alerts such as port 80 sweeps
from a proxy or that server X is forwarding mail and server X happens to be
a mail server, etc?  Intruvert has gotten significant feedback about
performing well with regards to false positives.  Here is the response from
the NSS Group:

SNIP
"Speed is not everything in the IDS world, of course.  Accuracy counts too,
and performance in our signature recognition tests is key here.  There is no
point achieving a big score if it is at the expense of accuracy, resulting
in a high number of false positives.  This certainly did not seem to be the
case here, and although IntruShield spotted every single one of our
exploits, we did not notice any serious false positives or mis-identified
attacks."
SNIP
 
Has anyone put IntruVert into full Prevention mode and what were the
effects?  I have not heard of anyone actually using IntruVert's prevention
mode, but mostly as an IDS. 

Boyle:  Maybe some of our customers who are using the product will reply.
Otherwise, Intruvert has and can provide references.  

While it seems that many IDS/IPS reviewers rank and measure finding attacks
high, it would seem equally if not, more important to rank false positives
high especially in Prevention mode.  Is there any reviewers that have
compared the false positives and false alarms of all the IDS/IPS products?
Has anyone here compared false positives of Introvert, Snort, Cisco,
RealSecure, etc?

Boyle: Agreed whole heartedly.  As throughput increases on NIDS, I would add
that it is not just prevention that is driving the need for lower false
positive rates.  The NSS Group conducted tests of several products that
covered many areas important to anyone considering NIDS.  One of which was
false positives.  I believe it costs $50 bucks but is worth every penny.

Thanks in advance!

________________________________
Samuel Cure
KPMG
Risk and Advisory Services (RAS)-Atlanta
Phone: 404.222.3043
Fax:    404.222.7740
Cell:    404.861.9436
mailto:scure () kpmg com
________________________________


****************************************************************************
*
The information in this email is confidential and may be legally privileged.
It is intended solely for the addressee. Access to this email by anyone else
is unauthorized. 

If you are not the intended recipient, any disclosure, copying, distribution
or any action taken or omitted to be taken in reliance on it, is prohibited
and may be unlawful. When addressed to our clients any opinions or advice
contained in this email are subject to the terms and conditions expressed in
the governing KPMG client engagement letter.         
****************************************************************************
*


-----------------------------------------------------------
ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
Learn why 70% of today's successful hacks involve Web Application
attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter 
Manipulation.
http://www.spidynamics.com/mktg/webappsecurity71

-----------------------------------------------------------
ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
Learn why 70% of today's successful hacks involve Web Application
attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter 
Manipulation.
http://www.spidynamics.com/mktg/webappsecurity71