Re: Tools comparison and evaluation question (AppScan)

[Tommy <tommy () providesecurity com>]

Serg,

As you are aware there is a never ending discussion about which scanner is
the best scanner for Application Scanning. Many times a persons opinion can
be tainted because they work for a VAR and the product is a partner of
theirs, the company published something saying they are the best to some
magazine OR numerous other reasons. Each scanner has its sweet spot for
detecting certain vulnerabilities. In order to show Due Diligence, What you
need to do is collect the Top 10 types of applications (Java, .NET, ASP,
PHP, Perl, ColdFusion) that you test the most. Contact the six vendors
below, then have a Bake Off between the 6 commercial tools.

NTOBJECTives NTOSpider 2.3,
SPIDynamics WebInspect 5.8, 
WatchFire AppScan 6, 
Cenzic Hailstorm 3.0,
WhiteHat Sentinel,
Accuntix 3.0

One thing you need to remember, an Application Scanner by itself will at
best discover 30% of the vulnerabilities. The other 70% are logical hacks.

****DO NOT JUST TEST AGAINST THE VENDOR TEST SITE****
****DO NOT JUST TEST WIZARD/AUTOMATION MODE THEY ARE NOT COMPLETE****

The areas you may find interesting and use as differentiators are:
*Number of False Positive Errors found
*Number of Positive False Errors found
*How the scanner handles authentication
*How well the scanner compensates for Error Handling
*Does the report provide an accurate enough fix to hand off to a developer
*Is the information in the fix report correct (You will see a lot of
problems with fix reports involving TomCat and many others)

***My favorite one was that NO Scanner found a Select Statement in a Hidden
Field. Can it be any more BASIC?

At the end of the month I am releasing a paper on my findings of the
scanners. The scanners benchmarked each scanner against 37 Applications
(Java, .NET, ASP, PHP, ColdFusion). It outlined the pros and cons of each
scanner, unique features of each scanner. The paper is not designed to BASH
Scanners and say "They Suck", but show the errors in scanners and how to
over come some of them, and many of the errors the people using the scanners
make.

Those of you that are attending the NY Metro Infragard Meeting in NYC
Sponsored by Cisco, will see key elements of the paper before it's
published.

The rest of you will have to wait and see it when I publish the site on
ApplicationScanner.net

Best of Luck,

Tom Ryan


-------------------------------------------------------------------------
This List Sponsored by: SpiDynamics

ALERT: "How A Hacker Launches A Web Application Attack!" 
Step-by-Step - SPI Dynamics White Paper
Learn how to defend against Web Application Attacks with real-world 
examples of recent hacking methods such as: SQL Injection, Cross Site 
Scripting and Parameter Manipulation

https://download.spidynamics.com/1/ad/web.asp?Campaign_ID=701300000003gRl
--------------------------------------------------------------------------