RE: Comparison report on web app security scanners

[<Holger.Peine () iese fraunhofer de>]

In response to the postings by Bogdan Calin of Acunetix and
Ory Segal of Watchfire (i.e. web security scanner vendors),
I agree with them that Webgoat is not a good benchmark
application, as are the other "web security training applications".

However, what's the alternative? Certainly not the vendor-controlled
and vendor-operated online test applications most tool vendors offer.
Maybe OWASP SiteGenerator can be used here once it is finished, provided

that all tool vendors can agree on what constitutes a "fair" application

(an agreement I would not bet my life on, however). However,
SiteGenerator
was not available when I did my tools comparison.

For the time being (or, more exactly, for the time of last fall when I
did my comparison), I did not see a better alternative than using
Webgoat (for its breadth of vulnerabilities) plus a more "typical", i.e.
production application. While I cannot disclose any details about that
other application (remember that the tools _did_ find some
vulnerabilities
in that one, and the operator of that application does not want to be 
connected in any way with those), you read read in my report that the
tools
did not do much different on that application - in both cases, their
performance left a lot to be desired.

If you consider buying a web app scanner to secure a certain application
of yours, I advise you by all means to try the scanner on your
application,
not on any arbitrary benchmark. Tool performance varies wildly with the
application, so pay close attention to your specific situation.

Best wishes for your tool activities,
Holger Peine

-- 
Dr. Holger Peine, Security and Safety
Fraunhofer IESE, Fraunhofer-Platz 1, 67663 Kaiserslautern, Germany
Phone +49-631-6800-2134, Fax -1299 (shared)
PGP key via http://pgp.mit.edu ; fingerprint is 1BFA 30CB E3ED BA99 E7AE
2BBB C126 A592 48EA F9F8

-------------------------------------------------------------------------
Sponsored by: Watchfire

Watchfire named worldwide market share leader in web application security
assessment by leading market research firm. Watchfire's AppScan is the
industry's first and leading web application security testing suite, and
the only solution to provide comprehensive remediation tasks at every
level of the application. See for yourself.
Download a Free Trial of AppScan 6.0 today!

https://www.watchfire.com/securearea/appscansix.aspx?id=701300000007t9c
--------------------------------------------------------------------------