Re: Web Application Source Vulnerability Scanners

["Jeff Williams @ Aspect" <jeff.williams () aspectsecurity com>]

I think all the tools mentioned in this thread are pretty good at what
they do, and getting better all the time.  I think they can automate a
small part of the problem very nicely.

In my experience, most of the big problems in web applications are logic
errors -- frequently flaws that result from poorly designed or
completely missing security mechanisms. These flaws are difficult to
scan for because every web application has different mistakes.  It is
not like scanning for vulnerabilities at the network layer where
everyone is running one of a handful of operating systems.

A perfect scanning tool would be able to find ALL possible variants of
the OWASP top ten web application vulnerabilities. I'd be willing to bet
that none of the existing tools can find all possible variants of ANY of
the top ten.  Again, no disrespect to the folks who have built these
tools.  The problem of finding security vulnerabilities in arbitrary
custom code is extremely difficult, especially from an external
perspective.

In my opinion, if you want to find flaws in custom web applications,
look at the code.  All the flaws are there in black and white.  If you
can't read the code, find someone who can.  A security code review is
the fastest way to find the most serious holes in your web application.

--Jeff

Jeff Williams
Aspect Security, Inc.
http://www.aspectsecurity.com



----- Original Message -----
From: Kevin Spett
To: webappsec () securityfocus com ; ory.segal () sanctuminc com ;
securityarchitect () hush com
Sent: Tuesday, March 04, 2003 2:22 PM
Subject: Re: Web Application Source Vulnerability Scanners


Moderator: As SA stated, this is delicate as it involves the discussion
of
commercial software produced by the companies that I (and Ory) work for.
However, I think that my points are valid and discussion-worthy.

First, there are other tools besides AppScan that know how to keep state
correctly.  WebInspect does.

Second, on to the aforementioned article.  The info security magazine
test
report is less than scientific, as it doesn't detail any of the exact
testing procedures, server configuration, give source code for the
applications, etc.  It does not allow anyone to duplicate or evaluate
its
findings.  Also, the test was financed and performed by a company that
makes
money by performing services that tools such as AppScan and WebInspect
are
designed to test.  Finally, the two authors of the article are not
well-known in the area of web application security.

Keep in mind that these are not accusations.  I am not alleging that the
test results were incorrect.  I am not saying that the authors are
unqualified.  I'm just saying that the test really doesn't really
provide
enough information for real technical discussion.  Thus, its findings
cannot
be "proved" either way.  I distrust that which cannot be proved.

I simply recommend that people who are interested in appraising the
quality
of web security tools, both free or commercial, make their own tests and
judgements, so that they can control every variable of the analysis.
This
will have to be the case until there are truly "open" evaluations that
are
not lacking in steps for reproduction.



Kevin Spett
SPI Labs
http://www.spidynamics.com/


----- Original Message -----
From: <securityarchitect () hush com>
To: <webappsec () securityfocus com>; <ory.segal () sanctuminc com>
Sent: Tuesday, March 04, 2003 11:48 AM
Subject: RE: Web Application Source Vulnerability Scanners


> I know this list doesn't cater for commercial tool discussions
specifically so choosing words carefully moderator ;-)
> To counter that you should look at the latest review of commercial
tools.
All failed pretty miserably and the general recomendation was to wait
until
the next generation of tools come out.
> http://www.infosecuritymag.com/2003/jan/cover.shtml
>
>
> On Tue, 04 Mar 2003 07:25:02 -0800 Ory Segal
<ory.segal () sanctuminc com>
wrote:
> > Hi,
> >
> > The problem with most open source tools is that they are very strong
> > in
> > CGI Scanning, but when it comes to mutating real HTTP requests,
> > and
> > testing the web application layer, they lack good engine features.
> > They
> > do not have features such as:
> > 1) Application level tests such as manipulation of : HTML form
> > parameters (SQL Inj., Buffer Overflows, Poison null byte, Format
> > strings
> > bugs, Cookies, HTTP Headers etc...)
> > 2) Automatic testing validation.
> > 3) Good reporting abilities
> > 4) Session management/Transient management - Keeping the scanner
> > 'in
> > session'. This gives you the ability to scan web applications that
> > force
> > you to login, and may kick you out of session, if you caused some
> > error
> > - I believe that most large web apps have this. I believe that
AppScan
> > is the only scanner to perform this action.
> > 5) Good performance
> > 6) Contstant updates.
> > 7) Logging of raw HTTP traffic
> > 8) The ability to easily implement new tests.
> >
> > -Ory Segal.
>
>
>
> Concerned about your privacy? Follow this link to get
> FREE encrypted email: https://www.hushmail.com/?l=2
>
> Big $$$ to be made with the HushMail Affiliate Program:
> https://www.hushmail.com/about.php?subloc=affiliate&l=427