What a security test should do?- from thinking about: Ethical Hacking Training

["Pete Herzog" <pete () isecom org>]

What does a pen test fail to provide?

I had to think about this for a little while because it's not so much to me
what someone needs to know to be a security manager, CISO, or security
consultant, but rather what do we expect from a security test?

I know what pen-tests have been used for but I think a lot of that is also
under-analyzing the results of a pen-tset.  As an auditor of pen-test
reports for some companies, I see many of these reports focusing on software
vulnerabilities, the occassional rooting of boxes, and the holy trilogy of
web app hacks (XSS, Command Injection, Buffer Overflows).  Most reports will
have a traceroute to each host in the network but not even say why or what
that is useful for.  So in the end these reports leave a lot of analysis up
to the client and if they are not capable of this kind of analysis, the
report has much less worth.

I have felt that security tests should do more. They should test
configurations and policies as well.  A test may tell you, for example,
about patch management, which department influences the company's Internet
presence, and if the firewall admin has top-level support or a policy to
follow regarding opening new ports.  All of these things may negatively
influence the strength of network security in ways that make it just as
vulnerable as a remote service exploit.

As Jeff mentions here, there is a lot more to network security than
pen-testing but for the most part, testing should be also able to verify
when the foundation is rotten.

So my question is, what parts of security can't be verified in a security
test?  No flames please-- I'm just trying to make the OSSTMM (osstmm.org)
better.

Sincerely,
-pete.

Pete Herzog, Managing Director
Institute for Security and Open Methodologies
www.isecom.org - www.osstmm.org
www.hackerhighschool.org - www.isestorm.org


> -----Original Message-----
> From: Jeff Shawgo [mailto:jeff.shawgo () verizon net]
> Sent: Tuesday, January 20, 2004 18:46 PM
> To: pen-test () securityfocus com
> Subject: Re: Ethical Hacking Training
>
> On the other hand, most people also forget that knowing how to
> perform a pen-test or exploit is only one very very tiny aspect
> of security.  The organization that has a solid policy,
> coordinated antivirus, well-managed firewalls, patch management
> policy, e-mail and web filtering, code review, and basic system
> hardening is likely to be many times more secure than the
> organization that focuses on *any* one individual's skill as a
> pen-tester.
>
> If the security foundation is rotten, it does little good to
> point out that the windows are unlocked.
>
> Pen-testing is important, but the basics need to be there first.
> That's the message most people are missing - probably because
> it's not as attractive.
>
> ~Jeff
>
> ------------------------------------------------------------------
> ---------
> ------------------------------------------------------------------
> ----------


---------------------------------------------------------------------------
----------------------------------------------------------------------------