RE: Of the three expensive vulnerability scanners

["Michael Silk" <michaels () phg com au>]

Hi Jeff,

        Surely though a package that looks for specific use of a library
could be better implemented with a simple question: Did you use
"User.IsInDomain(...) to authenticate..." (whatever the function is
called) ... ?

         I mean, this is something that at least _one_ of the developers
would _know_ ... Rather then an over-sight that they made ... ?

        Further (if I understand you right ...) it wouldn't catch people
using "custom" systems of authentication and not ones in-built in the
language (like I do ...).

-- Michael

-----Original Message-----
From: Jeff Williams [mailto:jeff.williams () aspectsecurity com] 
Sent: Tuesday, 23 November 2004 2:08 AM
To: Adam Shostack; webappsec () securityfocus com
Subject: Re: Of the three expensive vulnerability scanners

Totally agree that you need to find the right mix of methods for finding
vulnerabilities. You can test the running application with automated
scanning and manual penetration testing.  You can examine the source
with automated static analysis and manual code reviews.  The automated
methods have lots of misses (both false positives and missed negatives),
and the manual methods are only as good as the people involved.

I'm hoping for more products that assist human reviewers, rather than
attempting to automate the impossible.  Products that assist manual
reviewers in finding and diagnosing critical parts of the code and
configuration can be extremely effective.  Most products that attempt to
do everything just produce noise.

By the way, keep an eye out for new static analysis technologies that DO
find issues like missing security mechanisms. They know what the
libraries are for!

--Jeff

Jeff Williams
Aspect Security, Inc.
http://www.aspectsecurity.com

----- Original Message -----
From: "Adam Shostack" <adam () homeport org>
To: <ban.marketing.bs () hushmail com>
Cc: <webappsec () securityfocus com>
Sent: Sunday, November 21, 2004 2:46 PM
Subject: Re: Of the three expensive vulnerability scanners


> I know of companies that deploy millions of lines of new code
> annually.  (Both in house and outsourced code)  Deciding what to have
> an expert look at is hard and slow.  Adding any automation makes their
> experts more effective.
>
> So you need do decide between static testing, dynamic testing, or some
> mix.  Static testing is very good at finding some things, but not
> others.  It finds strcats, but doesn't find a lack of authentication.
> (I like to think of these as sins of commission vs. sins of
> ommission.)
>
> I'm not going to argue for or against the commercial dynamic test
> tools...I just don't know enough about them.  But dynamic testing is
> not fundamentally flawed, its a potentially useful part of a toolset.
> Would you not nmap and nikto boxes before they go out, just as a
> sanity check?
>
> Adam
>
>
> On Tue, Nov 16, 2004 at 06:14:28PM -0800,
ban.marketing.bs () hushmail com 
> wrote:
> | OK what am I missing here? Why use a fundamentlaly floored
> | technique for finding the issue? Why not look at the source? Its
> | pretty damn obvious where you are reading or writing unvalidated
> | data....please please no "source is not always available"
> | junk.....this is the web and 99% your looking at bespoke apps. You
> | have to ask or educate the client at worse.
> |
> | Its about time the industry started taking software security
> | seriously and continuing down this futile route of refining pen
> | testing techniques to make up for the obvious limitations of this
> | technique is not it IMHO.
> |
> | Newsflash - Most serious XSS issues in the real world are stored
> | not refelcted and unless you can trace data to the reflection point
> | this technique will NEVER find them !
> |
> |
> |
> | In-Reply-To: <003801c4c9c6$e5f39530$8d8606d1@rockstar>
> |
> | Jim,
> |
> | The problems you've mentioned with regard to the Cross Site
> | Scripting tests
> | point to a functionality area where the major players in the App
> | security
> | market need major improvement. As Jeremiah pointed out, the problem
> | is
> | broader than XSS policies alone, but it certainly affects them.
> |
> | One reason the XSS policies yield diminishing returns and are
> | poorly
> | organized in reports is due in part I believe to a lack of proper
> | detection
> | mechanisms. Both products use a plethora of fault injection
> | techniques, yet
> | neither seems sensitive to whether or not the injected script is
> | returned
> | within the context of the app's response in a form that is
> | executable by a
> | browser. As a result, when one form field is vulnerable to XSS, you
> | can get
> | into situations where virtually every XSS test returns with a
> | positive
> | detection.
> |
> | As you've no doubt noticed, each product checks for various kinds
> | of XSS,
> | some of these kinds are distinguished on the basis of the delimiter
> | that is
> | used. Despite the technical differences, each delimiter type has a
> | sophisticated name (i.e Double Quote Single Quote Bracket kung fu,
> | etc.)
> |
> | ">&lt;script ....
> | '>&lt;script ....
> | ">">&lt;script ...
> | <--&lt;script ...
> | <textarea>&lt;script ...
> | etc.
> |
> | While the main vulnerability condition is whether or not an
> | application will
> | "echo back" the script sequences, real problem is that the
> | different
> | delimiters are important because some will execute when returned by
> | the
> | application, and others will not, depending upon the HTML/Script
> | code of the
> | application. This is why it is important to audit the application's
> | logic,
> | but there really is no reason to test for 12 different types of
> | cross site
> | scripting scenarios using different delimiters and script types if
> | the
> | detection mechanism can't account for which sequences actually
> | yield results
> | that are executable.
> |
> | The optimal solution in my opinion would be to emulate a browser
> | and trap
> | for alerts (or other events) and then to organize the report data
> | based on
> | which delimiters successfully generated the desired pop-ups (or
> | whatever
> | event is trapped for). The rest could be classified as warnings.
> | This would
> | help to minimize the multiple alerting problems that plague the XSS
> | tests
> | and produce frequently confusing results. While this wouldn't fix
> | the
> | reporting problems, it would help to attenuate the signal.
> |
> | -tom
> |
> |
> |
> |
> |
> | Concerned about your privacy? Follow this link to get
> | secure FREE email: http://www.hushmail.com/?l=2
> |
> | Free, ultra-private instant messaging with Hush Messenger
> | http://www.hushmail.com/services-messenger?l=434
> |
> | Promote security and make money with the Hushmail Affiliate Program:
> | http://www.hushmail.com/about-affiliate?l=427 





**********************************************************************
This email message and accompanying data may contain information that is confidential and/or subject to legal 
privilege. If you are not the intended recipient, you are notified that any use, dissemination, distribution or copying 
of this message or data is prohibited. If you have received this email message in error, please notify us immediately 
and erase all copies of this message and attachments.

This email is for your convenience only, you should not rely on any information contained herein for contractual or 
legal purposes. You should only rely on information and/or instructions in writing and on company letterhead signed by 
authorised persons.
**********************************************************************