Firewall Wizards mailing list archives
Re: How automate firewall tests
From: Shahin Ansari <zohal52 () yahoo com>
Date: Fri, 18 Aug 2006 10:26:53 -0700 (PDT)
If you would, please send me a copy of the paper you mentioned. I do have a comment, please look below: "Marcus J. Ranum" <mjr () ranum com> wrote: Strabla Ruggero wrote:
What I need is someone that could tell me which type of tests you do on your firewalls and that you like too see automated
You've chosen a fairly interesting problem. What do you intend to measure about a firewall? It turns out that pretty much the only aspect of firewalls that the industry has figured out how to measure is performance - most notably thoughput and total concurrent streams. Of course, since a firewall is a _security_ device one would want to measure something about its security but it turns out that security is a rather elusive property. Testing a firewall with crafted packets will measure - something - but it may measure very wrong. After all, unless your packets are crafted to be indistinguishable from live application traffic, I'd argue that a firewall was not very good from a security standpoint if it let any of the packets through. Indeed, if all you're measuring is performance, the same applies - firewalls that do layer-7 processing (How can you call something that doesn't do layer-7 processing a "firewall"? But that's another question) will have different performance properties depending on the application mix and the layer-7 data going through, let alone whether the data is correct or not. There's a paper or two that might help you. One (search for "Ranum Kostic Molitor") is quite ancient, but the problem remains the same. Email me privately if you want a copy; I can see if I can find it. Another is a paper I did back in the NFR days on how to cheat on IDS benchmarks. It's highly relevant. http://www.mail-archive.com/firewalls () lists gnac net/msg22759.html is a repeat thread of this topic from 2002. See also: http://www.snort.org/docs/Benchmarking-IDS-NFR.pdf I am curious how above material is affected now that Vendors like Cisco implemented packet statful inspection. All the items regarding UDP, ICMP, and few others change. The doco above says no good firewall should allowe ICMP, but now Cisco claims they keep track of what ICMP requests went out and will only allow 1 reply. So this would be a valid test now ha? I would also add some tests regarding how well and fast the firewall handles VoIP traffic. What VoIP protocols they support. What is the throughput for such packets. Good luck; you've bitten off a huge problem. There have been any number of attempts at testing firewalls (and IDS) poorly; I've yet to see a test that's worth a pinch of sand. mjr. _______________________________________________ firewall-wizards mailing list firewall-wizards () listserv icsalabs com https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards --------------------------------- Stay in the know. Pulse on the new Yahoo.com. Check it out.
_______________________________________________ firewall-wizards mailing list firewall-wizards () listserv icsalabs com https://listserv.icsalabs.com/mailman/listinfo/firewall-wizards
Current thread:
- Re: How automate firewall tests, (continued)
- Re: How automate firewall tests Keith A. Glass (Aug 20)
- Re: How automate firewall tests R. DuFresne (Aug 23)
- Re: How automate firewall tests Jim Seymour (Aug 23)
- Re: How automate firewall tests haim [howard] roman (Aug 23)
- Re: How automate firewall tests sai (Aug 20)
- Re: How automate firewall tests Dave Piscitello (Aug 30)
- Re: How automate firewall tests Marcus J. Ranum (Aug 20)
- Re: How automate firewall tests StefanDorn (Aug 20)
- Re: How automate firewall tests Patrick M. Hausen (Aug 21)
- Re: How automate firewall tests Paul D. Robertson (Aug 21)
- Re: How automate firewall tests Patrick M. Hausen (Aug 21)
- Re: How automate firewall tests Paul D. Robertson (Aug 21)
- Re: How automate firewall tests Patrick M. Hausen (Aug 21)
- Re: How automate firewall tests Paul D. Robertson (Aug 21)