Re: Environment for testing WebApp Security Scanners

["Gerald Quakenbush" <geraldq () mastermindsecuritygroup com>]

See www.quakenbush.com for an App called MasterBugs - it may be useful for
your testing. I wrote the app (which is GNU licensed) as a companion to my
book.

I recently did a bake-off using the top three scanners / appsec toolkits on
the market. As *automated* scanners I was very disappointed. As toolsets in
the hands of a professional they have varying degrees of usefulness.

MasterBugs is "real", ie, it doesn't fake anything. It was originally a
proof-of-concept app for a real software package. I dumbed it down, added a
lot of bugs (some of the most fun I've had programming ...) and I use it as a
teaching tool. It is written in legacy ASP script and requires a SQL Server
v2000. There are hundreds of remotely exploitable vulnerabilities in it.

All of the automated scanners I tested failed to find even 10% of the flaws.

One advantage you might have (if you move quickly) is that I just released it
a couple months ago and the as of 3 weeks ago, the scanners didn't have
specific rules for this app.

Have fun...

-Gerald Quakenbush
Author of Web Hacker Boot Camp

Dean H. Saxe (dean () fullfrontalnerdity com) wrote:
> Just remember, some vendors put in signatures specifically for apps
> like this that they know will be tested.
>
> I spoke with some folks at Blackhat last week from one of the tool
> vendors that admitted they don't find all the vulnerabilities in
> HacmeBank.  Why?  Because bypassing the login form using SQL
> injection may require you to throw away the cookies which maintain
> the login attempts state information.  For instance, if you try SQL
> injection too many times and fail to login with those attempts, a
> cookie which controls the remaining number of login attempts will
> force all further attempts to fail.  Of course, a human would delete
> the login attempt counter cookie and solve the problem quite simply.
>
> I see this as a major weakness of these types of tools.  Developers
> store all kinds of crap in cookies, without a good analysis of the
> cookies by a human, how do we know when deleting this information
> will adversely affect the application's security?  This is not an
> edge case where such tools scan.  Its a very commonly revealed
> vulnerability in my web app pen testing and code review experience.
>
> (FWIW I work for Foundstone.)
>
>
> -dhs
>
> Dean H. Saxe, CISSP, CEH
> dean () fullfrontalnerdity com
> Here in America everything is bought and sold, you can get anything
> for little bits of gold.
> We'll rape the earth and ruin the air, cut down every tree from here
> to there.
>      -- Donna The Buffalo "America"
>
>
> On Aug 8, 2006, at 10:25 AM, Brokken, Allen P. wrote:
>
> > You might also consider looking at Foundstone's Hacme suite of
> > sites as a compliment to site generator.  I've found that in
> > testing scanners each methodology for building a site works
> > different "muscles" in the scanner and having a diverse back drop
> > to test against is important.
> >
> > Also, it has been useful to have someone unfamiliar with the
> > various sites and scanners being tested do the actual scanning.
> > I've found that having an intimate knowledge of the site and
> > scanner can boost individual performance drastically and compared
> > to the same person using different tools on a site they are
> > unfamiliar with.
> >
> > You might consider bringing in classmates who are technical, but
> > unfamiliar with the tools / sites to do at least one round of your
> > testing.  Most of the results/studies I've seen are done by highly
> > trained professionals.  However, most purchasers of scanners have
> > not risen to that level yet.  So a study along those lines would be
> > very useful.
> >
> >
> > Allen Brokken
> >
> > Information Security and Account Management - IAT Services -
> > University of Missouri -brokkena () missouri edu - (573)884-8708
> >
> > -----Original Message-----
> > From: René Palige [mailto:rwp () gmx de]
> > Sent: Monday, August 07, 2006 3:33 PM
> > To: webappsec () securityfocus com
> > Subject: Environment for testing WebApp Security Scanners
> >
> > Hi!
> >
> > I?m currently working on my bachelor thesis which is about the
> > development
> > of a testsuite for different Web Application Security Scanners. My
> > goal is
> > to provide an environment which can be used as a basis for testing and
> > evaluating the performance of the many tools already existing.
> > Consequently the main part of my work will be to implement
> > different types
> > of vulnerabilites in more or less realistic scenarios and with
> > different
> > characteristics. At the moment I?m planning to use OWASPs WebGoat
> > as some
> > kind of groundwork.
> > My questions:
> > Which "features" would you consider to be necessary or useful in this
> > context? And what basic requirements do you see which should be
> > met? Would
> > it be best to focus on "real-life scenarios"? Or rather to cover as
> > many
> > aspects of a special class of vulnerabilities as possible?
> >
> > Thanks in advance,
> > R. Palige
> >
> >
> >
> >
> > ----------------------------------------------------------------------
> > ---
> > Sponsored by: Watchfire
> >
> > Watchfire was recently named the worldwide market leader in Web
> > application security assessment tools by both Gartner and IDC.
> > Download a free trial of AppScan today and see why more customers
> > choose
> > AppScan then any other solution. Try it today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701500000008VnB
> > ----------------------------------------------------------------------
> > ----
> >
> >
> > ----------------------------------------------------------------------
> > ---
> > Sponsored by: Watchfire
> >
> > Watchfire was recently named the worldwide market leader in Web
> > application security assessment tools by both Gartner and IDC.
> > Download a free trial of AppScan today and see why more customers
> > choose
> > AppScan then any other solution. Try it today!
> >
> > https://www.watchfire.com/securearea/appscancamp.aspx?
> > id=701500000008VnB
> > ----------------------------------------------------------------------
> > ----
>
>
> -------------------------------------------------------------------------
> Sponsored by: Watchfire
>
> Watchfire was recently named the worldwide market leader in Web
> application security assessment tools by both Gartner and IDC.
> Download a free trial of AppScan today and see why more customers choose
> AppScan then any other solution. Try it today!
>
> https://www.watchfire.com/securearea/appscancamp.aspx?id=701500000008VnB
> --------------------------------------------------------------------------


-------------------------------------------------------------------------
Sponsored by: Watchfire

Watchfire was recently named the worldwide market leader in Web 
application security assessment tools by both Gartner and IDC. 
Download a free trial of AppScan today and see why more customers choose 
AppScan then any other solution. Try it today!
  
https://www.watchfire.com/securearea/appscancamp.aspx?id=701500000008VnB
--------------------------------------------------------------------------