RE: False Positives with IntruVert

["Alan Shimel" <alan () latis com>]

Bill

Its not often that I agree with the competition, but your email was one
of the most constructive, right on defense for IPS that I have seen.
Nice points!

alan 

Alan Shimel
VP of Sales & Business Development
Latis Networks, Inc.

303-642-4515 Direct
516-857-7409 Mobile
303-642-4501 Fax

www.stillsecure.com
Reducing your risk has never been this easy.
. . .
The information transmitted is intended only for the person
to which it is addressed and may contain confidential material.
Review or other use of this information by persons other than
the intended recipient is prohibited. If you've received
this in error, please contact the sender and delete
from any computer. 

-----Original Message-----
From: Bill Boyle [mailto:bboyle () intruvert com] 
Sent: Friday, March 28, 2003 1:18 PM
To: 'focus-ids () securityfocus com'
Subject: RE: False Positives with IntruVert 

OK.  I'll bite.  *courteous disclaimer: obviously from my email I work
for a
vendor.  Also, these are my personal opinions may or may not reflect
that of
my company.  That being said, the goal of this email is not to sell
anyone
product but to put into perspective the CURRENT value of IPS. 

This is not the first email to come across the list saying that IPS is
an
unusable product because of false positives.  What I have not seen on
this
list is the statement "IPS is not an all or nothing technology."  Nor
have I
seen the statement "I have recently tested and IPS solution and..."  An
IPS
solution, like any solution, is not a silver bullet solution.  There
isn't a
vendor out there who is claiming that their solution eliminates false
positives or is the end-all solution.  It is another tool.  

There are clearly attacks that will 100% not false positive.  My
question to
the community is: Why would you NOT block this traffic on your network?
We
have all been burned with false positives and I am sure that is where
the
mistrust originates.  But, a lot of technology has been put forth to
more
accurately identify attacks since IDS's inception.  

What I see as necessary for IPS to be useful TODAY is not zero false
positives.  It is performance, accuracy, and granularity of policy.

PERFORMANCE:  First and foremost, the box sitting on the wire must
process
the packets at wire speed and not drop ANY packets. It must also have
little
downtime (same expectations as my switch and router or firewall
infrastructure).  

ACCURACY:  Exemplified in Paul Schmehl's email, a signature alone is not
always accurate enough.  Correlation must be used to enhance the
accuracy.
For example, Intruvert uses Boolean logic in its signature set to
provide
more accurate detection.  Example: sig#1 in URI request AND (sig2 OR
sig3 in
body).

Not every signature is 100% accurate but it does not mean it is not of
use.
Example: a TCP connection on port 5432.  It may be backoriface or may be
a
random source port that happened to be a well know backoriface port.  It
will be a false positive but one that I would want to review.  The fact
that
I got this alert does not mean my IPS is worthless, it just means that I
will not choose to automatically block this event.  Other backoriface
attacks that have a lower false positive rate will (at MY choosing-not
the
vendor's).  

The ability to drop the single packet or single flow is necessary.
Broad
responses such as drop all port 80 to a specific server because  of a
web
attack might leave me open to spoofing and denial of service.

In every IDS product there are a range of signatures and anomaly alerts.
Some are dead accurate, others are suspicious but indispensable.  Events
like these need to be rated in the console: This is a HIGH severity
attack
but a MEDIUM chance that this is a false positive.  

GRANULARITY OF POLICY: In order for an IPS to be effective, the rules
must
be allowed to be applied with precision.  Example: Only this individual
Code
Red attack gets blocked if it is an inbound attack destined for IIS
machines.  The second Code Red Attack (also comprised of multiple
signature
matches) will alert only for any direction.  This precision is what
makes
IPS usable.  You do not want to say block ALL ATTACKS.  You could, but
you
won't want to.  A usable product will permit the flexibility in any
environment.

My 2cents.

--bill 
bboyle () intruvert com


I have also responded inline to the questions below: 


-----Original Message-----
From: Cure, Samuel J [mailto:scure () kpmg com] 
Sent: Friday, March 28, 2003 12:36 PM
To: 'focus-ids () securityfocus com'
Subject: False Positives with IntruVert 

Looking for some feedback on IntruVert.  I have a client that is
evaluating
IntruVert in the lab and has been getting a lot of false positives on
their
network.  They are afraid to put IntruVert into the IPS mode, of
actually
stopping traffic based on false positives.  Gartner Group has claimed
that
everyone is moving from Detection to Prevention, but if the underlying
technology has this many flawed signatures, I do not see how anyone can
confidently use it and start blocking all attacks. 

Boyle: Please contact me offline.  I would be interested in seeing what
the
false positives are.  Are they informational alerts such as port 80
sweeps
from a proxy or that server X is forwarding mail and server X happens to
be
a mail server, etc?  Intruvert has gotten significant feedback about
performing well with regards to false positives.  Here is the response
from
the NSS Group:

SNIP
"Speed is not everything in the IDS world, of course.  Accuracy counts
too,
and performance in our signature recognition tests is key here.  There
is no
point achieving a big score if it is at the expense of accuracy,
resulting
in a high number of false positives.  This certainly did not seem to be
the
case here, and although IntruShield spotted every single one of our
exploits, we did not notice any serious false positives or
mis-identified
attacks."
SNIP
 
Has anyone put IntruVert into full Prevention mode and what were the
effects?  I have not heard of anyone actually using IntruVert's
prevention
mode, but mostly as an IDS. 

Boyle:  Maybe some of our customers who are using the product will
reply.
Otherwise, Intruvert has and can provide references.  

While it seems that many IDS/IPS reviewers rank and measure finding
attacks
high, it would seem equally if not, more important to rank false
positives
high especially in Prevention mode.  Is there any reviewers that have
compared the false positives and false alarms of all the IDS/IPS
products?
Has anyone here compared false positives of Introvert, Snort, Cisco,
RealSecure, etc?

Boyle: Agreed whole heartedly.  As throughput increases on NIDS, I would
add
that it is not just prevention that is driving the need for lower false
positive rates.  The NSS Group conducted tests of several products that
covered many areas important to anyone considering NIDS.  One of which
was
false positives.  I believe it costs $50 bucks but is worth every penny.

Thanks in advance!

________________________________
Samuel Cure
KPMG
Risk and Advisory Services (RAS)-Atlanta
Phone: 404.222.3043
Fax:    404.222.7740
Cell:    404.861.9436
mailto:scure () kpmg com
________________________________


************************************************************************
****
*
The information in this email is confidential and may be legally
privileged.
It is intended solely for the addressee. Access to this email by anyone
else
is unauthorized. 

If you are not the intended recipient, any disclosure, copying,
distribution
or any action taken or omitted to be taken in reliance on it, is
prohibited
and may be unlawful. When addressed to our clients any opinions or
advice
contained in this email are subject to the terms and conditions
expressed in
the governing KPMG client engagement letter.         
************************************************************************
****
*


-----------------------------------------------------------
ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
Learn why 70% of today's successful hacks involve Web Application
attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter 
Manipulation.
http://www.spidynamics.com/mktg/webappsecurity71

-----------------------------------------------------------
ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
Learn why 70% of today's successful hacks involve Web Application
attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter 
Manipulation.
http://www.spidynamics.com/mktg/webappsecurity71


-----------------------------------------------------------
ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
Learn why 70% of today's successful hacks involve Web Application
attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter
Manipulation.
http://www.spidynamics.com/mktg/webappsecurity71