RE: RE : ICSA certified - better?

["Monkman, Brian" <bmonkman () icsalabs com>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

First off, I would like to thank the moderator of focus-ids for
allowing this discussion to happen here. I am cross posting this to
Firewall Wizards. Let's continue discussion there.

Criticism based on facts I am more then happy to discuss. I'm going
to assume you haven't yet completed reading of the criteria as I
suggested in my previous post.

I won't even try to suggest that vendors do not look at ICSA Labs
certification as a "marketing thing". After all - is marketing not,
in part, figuring out what your target customers want and ensuring
that they get it? It would be insulting to those on the list if I was
to seriously suggest that marketing *needs* are not a part of what
brings vendors to submit products to ICSA Labs for certification
testing. That said, I think you are doing everyone here a disservice
by suggesting that for some reason that it is a bad thing. 

Let's look at some facts:

1. Every product that has been submitted to ICSA Labs for
certification 
   testing against version 4.0 of the criteria has had issues come up
   during the first pass of testing. You heard me right - EVERY
product.

2. A number of products have had significant issues such as -
inability
   to stand up to simple trivial DoS attacks, susceptibility to FTP
   bounce attacks, passing of malformed or fragmented packets in
violation
   of the implemented security policy, and the list goes on. Some of
these
   products come from vendors that are well known in the industry.

3. A large number of corporations, agencies and governmental and 
   non-governmental organisations require ICSA Labs certification
*BEFORE*
   they will even consider a product.

4. A week rarely goes by here at ICSA Labs where we do not host a
visit from
   one of the groups listed in the point above. Visits that are
undertaken so
   that group can learn more about who ICSA Labs is, what we do and
how we do it.

5. Every product that has been granted ICSA Labs Firewall
Certification must
   remain here at ICSA Labs for subsequent testing. That subsequent
testing
   takes on two forms. First, products are re-tested on a periodic
basis to
   ensure the current shipping version of the product still adheres
to the
   current certification criteria requirements. Second, as exploits, 
   vulnerabilities or product issues become known and if they
intersect with 
   a criteria    requirement, that product will be subjected to a
spot check. 
   If this testing uncovers a criteria violation then the vendor will
be 
   REQUIRED to address the issue within a timeframe specified by ICSA
Labs. 
   If this does not happen then the vendor is at risk of having the
product 
   become de-certified. There have been a number of products that
have been 
   de-certified for this reason.

I could go on for a while more but this is supposed to be a list for
the discussion of IDS issues. This discussion would probably be more
appropriate in a forum devoted to firewall issues and discussions,
like Firewall Wizards for example.

If you are interested in facts and want to learn more about what we
do please feel free to contact me directly. Alternatively, there is
plenty of information on our website. A whitepaper discussion of ICSA
Labs Firewall certification testing and testing results can be found
at:

http://www.icsalabs.com/html/communities/firewalls/fwwhitepaper.pdf

Additionally, every certified product has a publicly posted lab
report that outlines the tests executed, issues found and what it
took to address them. You can get to these lab reports from the
certified products page found at:

http://www.icsalabs.com/html/communities/firewalls/newsite/cert2.shtml

Best regards,

Brian
 

- -----Original Message-----
From: ph03n1x [mailto:ph03n1x () gmx net] 
Sent: Thursday, August 05, 2004 5:58 PM
To: Monkman, Brian
Cc: focus-ids () securityfocus com
Subject: Re: RE : ICSA certified - better?

Brian,

I don't want to critisize your work or whatever. But the point is
ICSA certification is a nice thing to have but it doesn't grant too
much. 
It's more a marketing thing.

I have seen firewalls with ICSA which lack imho important features in
practical use. So it basically says not too much about quality it
just grants that some features are implented in some way which is
written in a nice paper. For the customer this certificate doesn't
say much it's more an interesting marketing thing for the selling
company ;)

my 2 bits



Monkman, Brian wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Julius,
>
> Your information is a bit inaccurate. ICSA Labs has *never* certified 
> only the GUI portion of the product.
>
> If you are interested in seeing what we actually do, our criteria is 
> available to all at:
>
> http://www.icsalabs.com/html/communities/firewalls/certification/crite
> ria/criteria_4.0.shtml
>
> You will notice that our testing is a bit more encompassing then GUI 
> testing. :-)
>
> Best regards,
>
> Brian
> - ----
> Brian Monkman
> Technology Programs Manager, ICSA Labs
> 1000 Bent Creek Blvd., Suite 200
> Mechanicsburg PA 17050
> Phone:717.790.8141  Fax:717.790.8170
> E-mail: bmonkman () icsalabs com
> AIM: bmonkman03   Web: www.icsalabs.com
> PGP Key ID: 0x7E54D5CD
>
> "The sole purpose of human existence is to kindle a light of meaning in 
> the darkness of mere being." - Carl Jung
>
>
>
> - -----Original Message-----
> From: Julius Detritus [mailto:julius.detritus () ifrance com]
> Sent: Wednesday, August 04, 2004 1:43 AM
> To: focus-ids () securityfocus com
> Subject: RE : ICSA certified - better?
>
>  
>
> > So basically you can certify the Firewall part of your product but
> > the   rest is not certified.
> >    
>
> ... And sometimes the only certified of your product is the GUI
>
>
>
> _____________________________________________________________________
> Envie de discuter en "live" avec vos amis ? Télécharger MSN Messenger 
> http://www.ifrance.com/_reloc/m la 1ère messagerie instantanée de 
> France
>
>
> - 
> ----------------------------------------------------------------------
> - ----
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from 
> CORE IMPACT.
> Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to 
> learn more.
> - 
> ----------------------------------------------------------------------
> - ----
>
>
>
>
> *** END PGP VERIFIED MESSAGE ***
>
> -----BEGIN PGP SIGNATURE-----
> Version: PGP 8.0.3
>
> iQA/AwUBQRD71KMpP5h+VNXNEQIungCdEnUiISmOJoc0KuaYPczw01K1FtUAn0bj
> 5pDIZxvN3zqKakuqAo0IWOve
> =Rl8B
> -----END PGP SIGNATURE-----
>
> *********************************************************************
> ** This message is intended only for the use of the intended
> recipient and  may contain information that is PRIVILEGED and/or
> CONFIDENTIAL.  If you  are not the intended recipient, you are
> hereby notified that any use,  dissemination, disclosure or copying
> of this communication is strictly  prohibited.  If you have received
> this communication in error, please  destroy all copies of this
> message and its attachments and notify us  immediately.
> *********************************************************************
> **  
>
>
> ---------------------------------------------------------------------
> -- ---
> Test Your IDS
>
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks
> from  CORE IMPACT.
> Go to
> http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
> ---------------------------------------------------------------------
> -- ---
>
>
> ---------------------------------------------------------------------
> -- ---------------------------- Von Panda Platinum Internet Security
> eingefügter Text::
>
> Wenn es eine unerwünschte Mail (SPAM) ist, klicken Sie auf den 
> folgenden Link um diese neu zu klassifizieren: 
> http://127.0.0.1:6083/Panda?ID=pav_1585&SPAM=true
> ---------------------------------------------------------------------
> -- ----------------------------
>
>
>  



-----BEGIN PGP SIGNATURE-----
Version: PGP 8.0.3

iQA/AwUBQROdNqMpP5h+VNXNEQIEeQCgrfYU3bPSvvieqo69Gux740WVWuAAn3Lu
6juzudPDmQqMI3Pd6fj6qGDv
=xCtI
-----END PGP SIGNATURE-----


***********************************************************************
This message is intended only for the use of the intended recipient and
may contain information that is PRIVILEGED and/or CONFIDENTIAL.  If you
are not the intended recipient, you are hereby notified that any use,
dissemination, disclosure or copying of this communication is strictly
prohibited.  If you have received this communication in error, please
destroy all copies of this message and its attachments and notify us
immediately.
***********************************************************************


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE
IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more.
--------------------------------------------------------------------------