Re: [PEN-TEST] VVIGILANTe Security Scanner

[Alfred Huger <ah () SECURITYFOCUS COM>]

On Sat, 2 Dec 2000, Rietveld, Peter wrote:


> A related company is evaluating security scanners for PenTesting. They have
> had a convincing marketing presentation of Vigilante. From their website
> *www.vigilante.com I 've gathered that they have somehow mixed:


Well, I had to bite on this one. So before I rant, please note that I am
not singling out VIGILANTe here. In fact I know very little about them
other than their advisories and their website.

>   a.. Fyodor nmap v2.53
>   b.. ISS Internet Scanner NT v6.1
>   c.. Linux traceroute v1.4a5
>   d.. NAI CyberCop NT v5.5
>   e.. SC Robert 3.0.1
>   f.. Slayer icmp v2.1
>   g.. VIGILANTe Exploit Arsenal v1.11
>   h.. VIGILANTe PortScanner v1.29
>   i.. VIGILANTe protocolid v1.26
>
> This product is supposed to catch something like 1000 security bugs, but ah,
> how many does ISS find, or cybercop?


Well to be blunt, none of the above find 1000 vulns - or even close
to it. One thing I learned from writing scanners is a simple universal
truth - marketers lie, *alot*. All of the scanner vendor marketers have a
curious grasp on mathematics, we used to call it 'check math'.

The idea is this: every scanner has a large number of checks which do not
pertain to vulnerabilities. Policy issues are an excellent example of
this, any given scanner can report on any number of policy issues via
registry access or by inferring things from exposed services (sendmail
banners et al.). Policy issues such as you are displaying banners and
perhaps you should not be or you allow javascript on your browser and
perhaps you should not.  These checks end up with impressive tallies. In
CyberCop Scanner pre 5.5 there were probably 600 of them. ISS was probably
at par.

The problem lies in that marketers take up these policy checks and add
them to real vulnerabilities (buffer overflows etc.) and you end up with
an inflated misleading number. I am sure for every 'We check for over 1000
vulnerabilities' statement there is an engineer cringing with shame - I
know for sure I did on more than one occasion.

On top of this, a large number of the checks in commercial scanners do not
work at all, or at least not every time. Scanners which stray outside the
standard NT auditing or port enumeration and OS ID'ing arena and attempt
to honestly audit DNS or LDAP or other complex protocols and services
things get dicy. It's not a reflection of poor code (although at times it
is) it's a practical reality that most of these items are very difficult
if not impossible to audit mechanically. Often to perform this work you
need a human with considerable skill (a dated concept I know..). This is
not to say I do not think these attacks can be automated I just think it's
outside of the scope of a scanner to perform.

And while we are on the subject I should say that the above applies to NID
systems as much if not more so.


> Well, all this means they run more than one box, or run *Nix stuff on NT, or
> vice versa. Anyway, I am just a bit curious. Anyone take a deeper look into
> their product? Is it useable?

Sure, why not. I think the idea of tying products together is a good one.
Consultants make a habit of it to make up for weak areas in competiting
products. I do not know anything about the VIGILANTe products but Nmap,
ISS and CyberCop are pretty decent. And they have Linux tracroute, which I
guess is pretty good...... Anyone have any idea why *that* made it into
marketing literature?

-al


"Vae Victus"
SecurityFocus.com