Re: OS XSS and SQL scanner

["Rory McCune" <rorym.forums () gmail com>]

<resend in plain text>
My 2p (UK) on this.

I'd agree with you in saying that the best results come from Code
Review/Threat Model or Manual Pen test, but you hit the nail on the
head with the phrase "talented testers/reviewers" who are
unfortunately, in my experience, in limited supply.  Also the cost
implications can rule out that type of testing for applications, so a
lot of the time you need to make the best of the limited time you've
got.

In my opinion Web applciation scanners function best as an adjunct to
manual testing, not as a replacement for it.  My experience is that
they are useful for running very large numbers of tests to get
coverage for things like SQL injection and XSS on every field in an
application but they don't find things like logic errors and
authorization problems reliably, which isn't too surprising as those
items tend to be application specific and therefore extremely hard to
code a generic test for.

The other point I'd make is that I think that the current generation
of web application scanners are best used by experienced web app.
testers as opposed to being used by developers/non-specialist
security-types as they need a fair amount of tweaking to get the best
out of (although it's worth noting that when I reviewed the apps.
earlier in the year it seemed that appscan was moving in that
direction, will be interesting to see how they go)

Unfortunately I'm not at BlackHat :O(

cheers



On 8/2/06, Dean H. Saxe <dean () fullfrontalnerdity com> wrote:
> Here, here, Arian.
>
> Let's see the web app scanner folks go up against a manual pen test
> and code review/threat model on a series of apps.  One caveat: the
> results must be open for review, which means publishing the results
> in an open forum for all to see.
>
> FWIW, I'm a former customer of SPIDynamics.  I have experience web
> app scanners in an enterprise environment along with pen testing and
> code reviews.  I have a good idea how things will shake out:  Web app
> scanners are inexpensive to run but don't find significant numbers of
> vulnerabilities.  Pen tests are a decent measure of security at a
> reasonable cost when performed my talented testers.  Code review &&
> threat model finds the most vulnerabilities at the highest cost when
> performed by talented reviewers.
>
> Will any web app scanner companies actually subject their scanners to
> such a bake off?  If not, how can we trust the marketing material?
> Was Gary McGraw right in calling these tools "badnessometers"?
>
> I'm at BlackHat all week.  Email me and we'll get together and chat.
> I'll be attending the WASC gathering at Shadow Bar tomorrow night.  I
> hope to see some of you there.
>
> -dhs
>
> Dean H. Saxe, CISSP,  CEH
> dean () fullfrontalnerdity com
> "[T]he people can always be brought to the bidding of the leaders.
> This is easy. All you have to do is to tell them they are being
> attacked, and denounce the pacifists for lack of patriotism and
> exposing the country to danger. It works the same in every country."
>      --Hermann Goering, Hitler's Reich-Marshall at the Nuremberg Trials
>
>
> On Aug 1, 2006, at 2:35 PM, Arian J. Evans wrote:
>
> >
> >
> >> -----Original Message-----
> >> From: Mandeep Khera [mailto:mandeep () cenzic com]
> >>
> >> I am sorry to hear that you perceive some problems with our
> >> product. We take pride in being the most accurate product
> >> with least amount of false positives in the industry. This
> >> has been proven in many bake-offs by customers and
> >> independent journalists.
> >
> > Hate to take this a little off topic, but do you have any facts
> > that can support or back up these claims? Any data produced by
> > anyone competent that speaks to your "false positives" and also
> > your "false negatives"?
> >
> > I have failed to read a review yet to date that contains useful
> > information. So far what I've read varies from useless data
> > organized around features like "reflective buttons" (e.g.-the
> > Acunetix review posted to this list written by some woman
> > who writes windows software articles) to the other extreme
> > of uninformed opinion and inability to keep features between
> > the products straight (secure enterprise computing review).
> > This includes infosec magazine and online reviews, bake-offs,
> > and Gartner-style evals. Every one I have read so far is garbage.
> >
> > Not one covers actual tests run & and the how & why around them.
> >
> > This situation is no doubt due to the utter lack of skill
> > and understanding of the subject on the part of the authors.
> >
> > However, I think all on this list would welcome information
> > of a high-quality nature regarding scanner quality, if you
> > have anything like that to point us at.
> >
> > -ae
> >
> >
> >
> >
> >
> > ----------------------------------------------------------------------
> > ---
> > Sponsored by: Watchfire
> >
> > Do you test web applications for XSS, SQL Injections, Buffer
> > Overflows,
> > Logical issues and other web application security threats? Why not
> > automate this work with Watchfire's AppScan, the world's leading
> > automated web application scanner. Download AppScan today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701300000008BP9
> > ----------------------------------------------------------------------
> > ----
> >
>
>
> -------------------------------------------------------------------------
> Sponsored by: Watchfire
>
> Do you test web applications for XSS, SQL Injections, Buffer Overflows,
> Logical issues and other web application security threats? Why not
> automate this work with Watchfire's AppScan, the world's leading
> automated web application scanner. Download AppScan today!
>
> https://www.watchfire.com/securearea/appscancamp.aspx?id=701300000008BP9
> --------------------------------------------------------------------------

-------------------------------------------------------------------------
Sponsored by: Watchfire

Do you test web applications for XSS, SQL Injections, Buffer Overflows, 
Logical issues and other web application security threats? Why not 
automate this work with Watchfire's AppScan, the world's leading 
automated web application scanner. Download AppScan today!

https://www.watchfire.com/securearea/appscancamp.aspx?id=701300000008BP9
--------------------------------------------------------------------------