RE: Application Assessment

["Brokken, Allen P." <BrokkenA () missouri edu>]

The most concrete evaluation criteria I've seen regarding evaluating the security of web applications is the PCI 
standard used by the VISA/MasterCard etc... in verfying site security.  By concrete I mean they have laid out a series 
of well described issues that need to be verified and specific items to check for related to application level 
security.  These are found in their auditors guide primarliy in section 6, but there are a few items elsewhere.  This 
covers the operational and development practice as well as the code as well.
 
http://www.usa.visa.com/download/business/accepting_visa/ops_risk_management/cisp_PCI_Security_Audit_Procedures_and_Reporting.doc?it=il|/business/accepting_visa/ops_risk_management/cisp_merchants.html|PCI%20Security%20Audit%20Procedures%20and%20Reporting
 
 
In building an independent evaluation criteria this would give a listing of the majority of checks that should be done. 
 Then the other criteria would be operational in nature including, but not limited to
 
Level of reporting
Which checks are done automatically versus manually
Bandwidth needed to use the tool
Expected load on the target system.
 
The thing about the PCI standard is that it has already been vetted through the Security professionals in the 8 
instituions that make up PCI and their whole goal is getting their customers to secure their sites.  We may 
individually differ on the finer points, but it is a highly comprehensive list.  
 
One other benefit of starting with the PCI standard as the basic matrix for evaluation criteria is the ubiquity of the 
standard.  If you are doing eCommerce you must comply.  I believe it would help "sell" the evaluation criteria to the 
community at large since there have to be literally thousands of auditors nationwide who have to assess the PCI 
compliance of systems.  Being able to use a PCI driven evaluation criteria when looking for tools would be a huge 
benefit.
 
There may be other similar standards to serve as a template, however using an existing procedural list that is used by 
a large segment of the community will only help the situation.
 

________________________________

From: Mark Curphey [mailto:mark () curphey com]
Sent: Thu 8/11/2005 2:21 PM
To: 'Jeremiah Grossman'
Cc: webappsec () securityfocus com
Subject: RE: Application Assessment



Criteria for an assessment tool should not be driven (or created) by tools
vendors but by must be created by an independent body. I know of enough
banks and large companies who would come together to define the criteria and
I heard about a NIST project that is relevant. NIST seems like the only
suitable candidate to me. This kind of credibility and independence IMHO is
what would produce credible results. 

I actually think its relatively easy to come up with a good set of criteria.
You need to mimic whats out there in the real world. Real world sites range
from small simple sites to large complex ones. Vulnerabilities range from
obvious simple ones to non-trivial complex ones. A testing framework needs
to mimic this and capture results in a repeatable and consistent and fair
manner. If everyone is testing against the same benchmark then results are
comparative anyways. Some tools will be better at some things that others.
That's valuable in itself.

If the car industry created safety standards then we would have rubber band
seat belts ;-)

-----Original Message-----
From: Jeremiah Grossman [mailto:jeremiah () whitehatsec com]
Sent: Thursday, August 11, 2005 2:28 PM
To: Mark Curphey
Cc: webappsec () securityfocus com
Subject: Re: Application Assessment

Not withstanding the mischaracterization of WASC (www.webappsec.org), which
is the consortium  I assume you were referring to, web application scanner
performance reviews would be a good thing for the community. In fact at
Black Hat I was speaking to a couple of the scanner vendors about doing
exactly that. The response I got was positive.

The fundamental challenge is developing a fair and balanced criteria in
which to test the products. Web application scanners of the open source and
commercial variety have a largely differing features sets, including
vulnerability identification capabilities. None of them are closely
comparable and this inevitably skews results since there's no baseline. No
one wants to be treated unfairly by someone publishing negative and biased
performance reviews. I believe this is the primary concern on why the trial
agreements prevent publishing performance results.

This isn't to say the initiative can't be done and the vendors don't want it
to be done, but the interested product participants would first have to work
together to develop a fair testing criteria. 
Without they're buy-in its never going to work.  WASC has a strong working
relationship with many web application security  industry experts and
vendors to make this possible.

In the web application firewall (WAF) world, WASC has begun this
process:

  "Web Application Firewall Evaluation Criteria" Project (WAFEC) Led by Ivan
Ristic, author of "Apache Security" and Mod_Security
http://www.webappsec.org/projects/waf_evaluation/

Top vendors and experts are working together to develop the industry
standard testing criteria for evaluating the quality of web application
firewall solutions. The expectation is that anyone would be able to use the
criteria to evaluate a WAF product in a consistent manner. Whether is be a
customer, professional reviewer, vendors, consultant, etc.

I expect the web application scanner guys to follow suit, its really just a
matter of when.


Regards,

Jeremiah Grossman-


---------------------------------------------------------------------
The Web Security Mailing List
http://www.webappsec.org/lists/websecurity/

The Web Security Mailing List Archives
http://www.webappsec.org/lists/websecurity/archive/










On Aug 11, 2005, at 8:52 AM, Mark Curphey wrote:

> Seems like it would be pretty valuable to publish an independent (not
> by the vendors or the vendors consortium) review of performance the
> web app scanners.  Last time I looked the trial agreements prevented
> publication of comparisons and results. I know of a few magazines that
> would be happy to publish the results and I would volunteer to
> organize the testing.
>
> -----Original Message-----
> From: Ory Segal [mailto:osegal () watchfire com]
> Sent: Thursday, August 11, 2005 6:16 AM
> To: goenw
> Cc: pen-test () securityfocus com; Webappsec
> Subject: RE: Application Assessment
>
>  Hi,
>
> You should also check: http://www.webappsec.org (Web Application
> Security
> Consortium)
>
> With regards to utilities, you can download the free Watchfire
> Powertools (HTTP Proxy, HTTP request editor, etc.), here's the link:
> http://www.watchfire.com/securityzone/download/default.aspx
>
> At the same link, you can also download eval versions of Watchfire's
> AppScan product (An automated application security scanner).
>
> You can also find basic and advanced whitepapers on the subject at:
> http://www.watchfire.com/news/whitepapers.aspx
>
> -Ory
>
>
> -----Original Message-----
> From: Glyn Geoghegan [mailto:glyng () corsaire com]
> Sent: Thursday, August 11, 2005 4:48 AM
> To: goenw
> Cc: pen-test () securityfocus com; Webappsec
> Subject: Re: Application Assessment
>
> On 8 Aug 2005, at 12:53, goenw wrote:
>
>
> > Hi,
> >
> > anybody have experience with application assessment ? I am a network
> > guy, dont know much about the apps PT.
> > 1. is there any tools that allow me to do the assessment throughly ?
>
> If you're talking web-applications, check out www.owasp.org for a
> wealth of information on the subject.  You may also want to take a
> look at the webappsec mailing list at www.securityfocus.com.
>
> Typically, the kind of tools you'll need are the personal-proxy
> category, allowing you to intercept and modify communications between
> the client and server - see Paros Proxy, Odysseus and Burp Proxy, for
> example.
>
> There are fully automated tools, but in my personal experience the
> manual approach has worked more effectively.
>
> Fat client/binary assessment is a slightly different (and arguably
> more
> complex) beast, and probably off-topic for this list.
>
>
> > 2. should i have external party conduct this, what are the things i
> > should expect from them (success criteria) ?
> > any comments are appriciated.
>
> That depends on how confident you are with your abilities, the drivers
> for the assessment and a wealth of factors.  Normally, some coding or
> development background is essential to identify and understand
> potential vulnerabilities.
>
> Check out www.application-testing.com for our guide on the world of
> Application Security Assessments.
>
> --
> -------------------------------------------------------
> G l y n   G e o g h e g a n                   BSc, ARCS
> Principal Consultant                       Corsaire Ltd
> 3 Tannery House, Tannery Lane
> Send, Surrey, GU23 7EF, UK      UK: +44 (0)1483 226 000
> http://www.corsaire.com        Fax: +44 (0)1483 226 001
> -------------------------------------------------------
>
>
>
>
> ----------------------------------------------------------------------
> ------
> --
> FREE WHITE PAPER - Wireless LAN Security: What Hackers Know That You
> Don't
>
> Learn the hacker's secrets that compromise wireless LANs. Secure your
> WLAN by understanding these threats, available hacking tools and
> proven countermeasures. Defend your WLAN against man-in-the-Middle
> attacks and session hijacking, denial-of-service, rogue access points,
> identity thefts and MAC spoofing. Request your complimentary white
> paper at:
>
> http://www.securityfocus.com/sponsor/AirDefense_pen-test_050801
> ----------------------------------------------------------------------
> ------
> ---