Firewall Wizards mailing list archives
Re: How automate firewall tests
From: "Marcus J. Ranum" <mjr () ranum com>
Date: Thu, 17 Aug 2006 13:33:58 -0400
Strabla Ruggero wrote:
What I need is someone that could tell me which type of tests you do on your firewalls and that you like too see automated
You've chosen a fairly interesting problem. What do you intend to measure about a firewall? It turns out that pretty much the only aspect of firewalls that the industry has figured out how to measure is performance - most notably thoughput and total concurrent streams. Of course, since a firewall is a _security_ device one would want to measure something about its security but it turns out that security is a rather elusive property. Testing a firewall with crafted packets will measure - something - but it may measure very wrong. After all, unless your packets are crafted to be indistinguishable from live application traffic, I'd argue that a firewall was not very good from a security standpoint if it let any of the packets through. Indeed, if all you're measuring is performance, the same applies - firewalls that do layer-7 processing (How can you call something that doesn't do layer-7 processing a "firewall"? But that's another question) will have different performance properties depending on the application mix and the layer-7 data going through, let alone whether the data is correct or not. There's a paper or two that might help you. One (search for "Ranum Kostic Molitor") is quite ancient, but the problem remains the same. Email me privately if you want a copy; I can see if I can find it. Another is a paper I did back in the NFR days on how to cheat on IDS benchmarks. It's highly relevant. http://www.mail-archive.com/firewalls () lists gnac net/msg22759.html is a repeat thread of this topic from 2002. See also: http://www.snort.org/docs/Benchmarking-IDS-NFR.pdf Good luck; you've bitten off a huge problem. There have been any number of attempts at testing firewalls (and IDS) poorly; I've yet to see a test that's worth a pinch of sand. mjr. _______________________________________________ firewall-wizards mailing list firewall-wizards () listserv icsalabs com https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
Current thread:
- How automate firewall tests Strabla Ruggero (Aug 17)
- Re: How automate firewall tests Marcus J. Ranum (Aug 17)
- Re: How automate firewall tests Durga Prasad (Aug 18)
- Re: How automate firewall tests Marcus J. Ranum (Aug 18)
- Re: How automate firewall tests Isaac Van Name (Aug 20)
- Re: How automate firewall tests Marcus J. Ranum (Aug 20)
- Re: How automate firewall tests Tim Shea (Aug 21)
- Re: How automate firewall tests Paul D. Robertson (Aug 21)
- Re: How automate firewall tests ArkanoiD (Aug 21)
- Re: How automate firewall tests Marcus J. Ranum (Aug 21)
- Re: How automate firewall tests Chris Blask (Aug 22)
- Re: How automate firewall tests Patrick M. Hausen (Aug 22)
- Re: How automate firewall tests Durga Prasad (Aug 18)
- Re: How automate firewall tests Marcus J. Ranum (Aug 17)