Snort mailing list archives
Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ?
From: Jason Brvenik <jasonb () sourcefire com>
Date: Mon, 9 May 2011 10:31:22 -0400
On Mon, May 9, 2011 at 7:50 AM, firewalZ <firewalz () gmail com> wrote:
I personally am a bit sceptical of NSS and other similar so-called third party tests. From a user perspective, you can not get any information from them without paying big $. From a vendor perspective, you cannot participate without paying big $ (i have heard this same story from more than one vendor). Smells a bit like a racket to me.
The NSS tests are NOT vendor funded, this is why consumers have to pay for them. The reality with the tests is that they are comparative, all vendors have a chance to submit to the test, some are submitted by their customers, and some don't do it at all. They are not like the tests you pay a lab to create that result in a favorable review. Feel free to contact NSS with questions as I've found them to be fairly open. They also publish the full test methodology for all of their tests so you can evaluate them and comment.
Bottom line is that if you are able, build a small test lab (many open source options), look into getting demo loaners from various vendors and test for your self. Understand your environment, potential attack entry points and unique vulnerabilities. I feel this would be a far greater use of the time and money that an NSS report would cost.
I've seen lots of "evaluations" in my time and very few can stand up to basic scrutiny. I've seen tests designed to favor friends and tests that are so incompetent a router could pass. I've seen huge companies fumble testing and small companies get close. It is certainly not an easy thing. Doing a real network lab eval that is consistent, measurable, representative, and understandable is very difficult. Reality. I'll line my systems up against anything else available and am so confident in the people, technology, and hardware that I'll bet that in a fair test I win hands down every time.
FirewalZ On Thu, May 5, 2011 at 12:18 PM, Crusty Saint <saintcrusty () gmail com> wrote:Hi, For a while now i've been stumped by the NSS Labs IPS resulsts for checkpoint during the latest ISP Test. http://www.checkpoint.com/campaigns/intrusion-prevention-system/index.html http://www.checkpoint.com/images/campaigns/intrusion-prevention-system/nss-block-rate-ips.png for the picture which dates January 11th 2011 Now my question is to what extent does this claim hold any water ? I've seen and evaluated this recommended profile and it is sparse. IMHO not really what i would take for a profile that realistically holds back 97.3% of all tests. So to make things worse for this intermediate IPS Engineer i've laid my hands on the NSS Labs testing methodology to better understand if checkpoint either tweaked it's profile to nss-labs testing or they really did deliver close to the Holy Grail for what IPS's are concerned. I'm begging for your input and knowledgeable comments. Such trickery bothers me. And if it is not trickery i definitely need to soup up my skills and mojo fast. Best Regards, S-C -- - - - Security Engineer - Tags: Analyst Systems Security Linux Firewall Network Web Troubleshooting - If you think I deserve a rant, write me off-list ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
-- Regards, Jason. ------------------------------------------------------------------------------ WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
Current thread:
- NSS Labs : CheckPoint 97.3% recommended profile hoax ? Crusty Saint (May 05)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Joel Esler (May 05)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? firewalZ (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Nigel Houghton (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? beenph (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Martin Holste (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Crusty Saint (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Nigel Houghton (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Jason Brvenik (May 09)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Crusty Saint (May 09)
- <Possible follow-ups>
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Rick Moy (May 10)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Martin Holste (May 10)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Seth Hall (May 10)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Paul Halliday (May 10)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Martin Holste (May 10)
- Re: NSS Labs : CheckPoint 97.3% recommended profile hoax ? Crusty Saint (May 11)