Penetration Testing mailing list archives
RE: False-negatives in several Vulnerability Assessment tools
From: "Craig H. Rowland" <crowland () cisco com>
Date: Thu, 17 Apr 2003 12:28:43 -0500
My current employer, which is a Fortune 10 company, shall be referred to as "Ralph Co." I've been with Ralph Co for 2 years now. Our security there is relatively pathetic. I have had to go to upper managment because our security manager will run a scan at random and decide a given service needs to be terminated because the scanning tool that he's demo-ing that week says that it's a "critical vulnerablity". I have had to try to explain to him several times that he pays us a lot of money to exercise our professional judegement in verifying what is and is not a real vulerablity. His answer is that "The tool says so, so it must be."The nadir of this process was him insisting that we shut down a "Code Red Infected Server". Too bad it turned to out be a developers Apple iBook. My point with all this is what you do with the scans AFTER you run them. If you want intelligent analysis of the report, you get a security professional that knows how to check things manually and knows when output from the scanner looks dubious. Any reasonably intelligent person can operate the scanner software and print out the report when its done. The skill and expertise comes in interpreting the output and making meaningful suggestions that actually improve security.
Exactly. When you go to the hospital for a broken bone you have a X-Ray technician operate the machine, and an experienced radiologist who interprets the results. They don't simply hand you the X-Ray for personal interpretation and the bill. This is an important point that is frequently overlooked. I've seen a number of audits that were paid for by customers and consisted of nothing more than a nicely bound printout of a commercial scanner with almost no interpretation. Personally, I think this is a serious breach of responsibility. The results of a scanner can be misleading if you don't have a good knowledge of common vulnerabilities, commonly affected hosts, and patterns indicating misuse. Expecting a scanner alone to identify 100% of all threats is not practical for several reasons: 1) The author of the vulnerability check may have written it incorrectly. Or, more likely, it worked in their testlab environment but failed out in the field for a variety of reasons. 2) Performing an exhaustive scan against all the systems in a large enterprise is usually not feasible due to network constraints, stability of the backbone and scanned systems, and the dynamic nature of network deployments (wireless, DHCP, etc.). 3) The scanner does not have an internal view of the host being audited and can miss critical mis-configurations that result in an insecure setup, but appear "secure" from the outside with automation. I guess my point in all this is that proper interpretation of security tool results is critical. As much as the security industry would like to have the software do everything for the inexperienced user, it just isn't practical or advisable given the nature and seriousness of this business. -- Craig Opinions are my own. There is no endorsement of the (random) advertisement appended to this message. --------------------------------------------------------------------------- Attend Black Hat Briefings & Training Europe, May 12-15 in Amsterdam, the world's premier event for IT and network security experts. The two-day Training features 6 hand-on courses on May 12-13 taught by professionals. The two-day Briefings on May 14-15 features 24 top speakers with no vendor sales pitches. Deadline for the best rates is April 25. Register today to ensure your place. http://www.securityfocus.com/BlackHat-pen-test ----------------------------------------------------------------------------
Current thread:
- False-negatives in several Vulnerability Assessment tools Nicolas Gregoire (Apr 07)
- <Possible follow-ups>
- Re: False-negatives in several Vulnerability Assessment tools Muhammad Faisal Rauf Danka (Apr 16)
- Re: False-negatives in several Vulnerability Assessment tools R. DuFresne (Apr 16)
- Re: False-negatives in several Vulnerability Assessment tools Jimi Thompson (Apr 17)
- RE: False-negatives in several Vulnerability Assessment tools Craig H. Rowland (Apr 17)
- Port Scanners / Sniffers Review Sam (Apr 24)
- Re: Port Scanners / Sniffers Review cdowns (Apr 24)
- Re: Port Scanners / Sniffers Review Mary-RR (Apr 24)
- Re: Port Scanners / Sniffers Review Paul Vlissidis (Apr 27)
- Re: Port Scanners / Sniffers Review Philippe Biondi (Apr 30)
- Re: False-negatives in several Vulnerability Assessment tools R. DuFresne (Apr 16)