RE: (OWASP Web App Tool Project) Tools comparison and evaluation question (AppScan)

["arian.evans" <arian.evans () anachronic com>]

> -----Original Message-----
> From: Erwin Geirnaert [mailto:egeirnaert () securityinnovation be] 

> I hope that the project at Owasp about the web app scan 
> market (is it a project or an individual initiative?)

It is both. :) Dinis Cruz is writing a synthetic benchmarking (and learning)
tool over at OWASP.net, and I have some synthetic (and real) apps that
I look forward to working with Dinis to assemble in a manner people
can use to evaluate tools themselves.

> is able to shed some light on the real power of commercial tools.

I started the "project" because there was very little info out there,
and every magazine/online review I've read is low-quality. They vary
from useless (no details, no metrics) to inaccurate...like the Secure
Enterprise Magazine review from 05 where they can't even keep the tool
features straight, and no mention of bugs that existed at that time.

A significant problem is that there were/are no standard or clear
definitions for threats, attacks, weaknesses, and vulnerabilities.
The OWASP T10 does not currently distinguish, and the WASC 'threat
classification' is really an attack-matrix by any threat-modeling
system definition. No 'taxonomy of testing' exists, and no clear way
to categorize and organize the results exists.

These deficiencies have became the more interesting problem for me.

I just finished attempting to benchmark 15 tools, from automated
to manual fault-injection type tools, to two source code analyzers,
on six different web-based applications. It turned out to be far more
exhaustive than I expected, in the end I had to create two PoC (proof
of concept) point-apps, and I got through less than half of what I
had intended to due to time and complexity.

It was a great lesson on "how to go about this", which up until now
has been unforgivably unsystematic. My interest moving forward is
to focus on classification and definition, and a methodology for
people to self-evaluate (with tools like SiteGenerator), or at least
analyze the quality of evaluations done by magazines/third-parties.

> I can imagine that when you need automated assessment tools and
> only can rely on Google or banners on security sites (or even
> mailing list adds :)) to learn about these products, you don't know
> what to choose.

Yes, this is the problem, but there is such a wide variance I think
this area is impossible to give a clear "editor's choice award".


Arian













-------------------------------------------------------------------------
This List Sponsored by: SpiDynamics

ALERT: "How A Hacker Launches A Web Application Attack!" 
Step-by-Step - SPI Dynamics White Paper
Learn how to defend against Web Application Attacks with real-world 
examples of recent hacking methods such as: SQL Injection, Cross Site 
Scripting and Parameter Manipulation

https://download.spidynamics.com/1/ad/web.asp?Campaign_ID=701300000003gRl
--------------------------------------------------------------------------