Review of Owasp-London Chapter meeting on WAF (Web Application Firewalls)

[Dinis Cruz <dinis () ddplus net>]

Hello

Here is my brain dump about what happened (from my point of view of 
course) last Tuesday (25th April) on the Owasp London Chapter meeting 
(hosted on a central London pub).

As you can read here 
(http://sourceforge.net/mailarchive/forum.php?thread_id=10209024&forum_id=40284), 
here 
(http://sourceforge.net/mailarchive/forum.php?thread_id=10232868&forum_id=40284) 
and here 
(http://sourceforge.net/mailarchive/forum.php?thread_id=10239385&forum_id=40284) 
the theme of the night was "Web Application Firewalls (WAF): Where do 
they add value and who should be using them". After a public invite for 
participation, 4 vendors committed to come in an present their WAF 
appliances: F5, Imperva, Breach and Fortify (I should actually say 3 WAF 
Appliance vendors since Fortify presented a similar tool with a 
different approach).

My objective with this presentation was to go beyond the Marketing BS 
and discuss the type of issues that these tools:

   - are able to defend out of the box (i.e. with minor configuration's 
by the user)
   - are able to defend with some customization
   - are NOT able to defend

I was also very interested in the usability of the tools and the user 
experience of the WAFifyed website  (i.e. what happens when a potential 
attack is detected).

To level the playing field, each vendor was given a vulnerable website 
that they needed to protect. This website was Foundstone's HacmeBank 
v2.0 (the original plan was to use Owasp's SiteGenerator but some 
vendors had some problems in getting it to work (prob with windows 2003 
SP1 I think), so due to time constrains we used the new version of 
HacmeBank (2.0) which is very stable and feature (i.e. vulnerability) 
rich :)

To give an extra humph to the event (and to justify the tight deadlines) 
I added the element "Hacme Bank is under attack, here are the exploit 
vectors being used, your mission is to patch this system as soon as 
possible and stop the attackers". It should be noted that the vendors 
did not had a lot of time in their hands (from receiving the list of 
vulnerabilities to patch to the event).

For reference, here is a PDF (created using the new Owasp PenTest 
Reporter tool) describing all vulnerabilities: 
"http://209.97.215.160/install/Hacme Bank Pentest Report (created by 
Owasp PenTest Reporter) - ).pdf" .

Now that the scenario is set, here are my brain dump of the night:

   1) Firstly I am happy that the 4 vendors made it, that we had a good 
number of attendees (although not all Owasp London Chapter members :) 
and that the  F5 sponsorship went ok (with paid: pub dedicated to event, 
plasma screen, open bar and food)

   2) The audio recording also went ok, so give me a couple days and I 
will upload it to a server on MP3 format (need to do some audio 
normalization and minor editing)

   3) On the presentations it self (in the order they were made):
         a) F5
              - Brought their WAF Appliance (1U Rack) and did a live 
presentation
              - Showed several demos on the HacmeBank vulnerabilities 
(XSS, SQL Injection, Cookies)
              - Discussed briefly some advanced options that their 
product has: Leveraging the BigIP scripting capabilities, protecting 
admin pages by forcing a certain 'navigation path'
               - Maybe because this was the first presenter, but they 
really run out of time (the 20m minutes allocated were religiously 
enforced). I did felt that the presenter at times went over the top on 
the explanations given, and spent too much time on simple stuff. 
Basically I would had preferred if he gave the presentation at twice the 
pace covering twice the material.
         b) Imperva
              - Brought their WAF Appliance (1U Rack) and did a live 
presentation
              - Similar to F5, showed several demos on the HacmeBank 
vulnerabilities (XSS, SQL Injection, Cookies) covering roughly the same 
material
              - Shown very powerful functionality on their product, 
namely the Forms / Cookie rules creation user interface
              - Had several problems during the presentation (e.g. some 
demos that didn't work)
         c) Breach
              - No WAF Appliance, videos shown instead
              - A couple too many slides for my taste, and not enough 
demos.
              - That said, I liked Ofer style and he has clearly a very 
good technical understanding of the WAF world (he is also the organizer 
of the Owasp Chapter in Israel)
              - I didn't particularly agree with the concept of "Dinis 
WAF Requirements", which imply that what I was asking the WAF vendors is 
very advanced, difficult and has very little customer demand.
         d) Fortify
              - Fortify didn't present a WAF Appliance. They took the 
concept of WAF and applied it to the actual targeted application. Steps 
used by their product: 1) reverse engineering the J2EE application, 2) 
identify the attack surface, 3) identify insecure coding practices, 4) 
apply 'security protection layers' were necessary, and 5) monitor and 
report malicious activity. Due to the fact that their .Net product is 
still under development, Fortify's demo was based on the Owasp Web Goat 
tool.
              - I was quite impressed with their solution and can see 
real-live cases where it could be very effective (especially since some 
of the security vulnerabilities that I want to address via a WAF 
solution can only be made at the Application layer)
              - The main problem with this solution is that it requires 
a binary change/redeployment of the targeted application, and it still a 
very young product (e.g. not used by a large number of clients to 
protect mission critical web applications)

[this next set of comments apply mainly to the 3 WAF vendors]

  5) Coming back to the WAF Appliance presentations, there are a couple 
of comments that I have to make:

       a) The presentations were all very BASIC (as in Simplistic). 
Come on!!!! doing an SQL injection or cookies demo using Paros? That is 
'so' last century. I was expecting at least some Unit Tests or custom 
use of Web Application Security Scanners. But Paros? (or for that mater 
Web Scarab, Fiddler, Firefox/IE Tamper request, @Stake Web Proxy, 
etc...). My problem here is not that such tool(s) were used, but the 
fact that they don't allow for the automated test of the numerous 
variations that exploits like SQL Injection have. Remember that the 
WAF's must be able to detect ALL (or most) types of SQL Injections (not 
only the ones with a quote). Think of the problems that IDS have in 
detecting RPC traffic, the WAF will have exactly the same types of issues.

       b) I might be wrong on this one, but I do get the feeling that 
these WAF vendors have a good understanding of 'data validation' attacks 
( SQL Injection, XSS, Cookies, etc) but fail to grasp the security 
implications of (for example) application logic attacks (like the one in 
Hacme Bank where user A is able to access data from user B). More 
worryingly, they seem happy with this situation since (from their point 
of view) their clients don't need that functionality.

      c) Where are the published independent security reviews of these 
products? I find amazing that vendors that are selling a 'security 
product', e.g. a software application (WAF) that protects other software 
applications (Websites), do not understand the value of hiring 
independent 3rd party security companies to perform source code security 
audits to their products (note that the final results of these audits 
must be published and made available to clients). As discussed during 
the panel, it is probably impossible to create bug/vulnerability free 
applications, but to NOT perform independent security audits to their 
code is crazy. Since these vendors are still in the 'Functionality Arms 
Race' phase of their products. Basically, the development teams are more 
focused on features, performance and user experience than on Security 
(and I don't have to tell you how 'secure' apps developed like this tend 
to be :). Maybe the solution is to put a WAF protecting a WAF protecting 
a WAF protecting a website :). Note to vendors: If am am wrong in this 
comment, feel free to prove me wrong and publish the security audits 
performed on your current product(s).

      d) another area which I still think the WAF don't get it, is the 
fact that no solution allows for the easy manipulation of the data being 
analyzed (both at input and at output). At the moment you only have two 
choices: 1) let the request go, 2) block the request and either show a 
custom error page or logout the user. This is to radical. We need the 
ability to change the contents of form fields (or page contents) 
dynamically. This will be the only way that some vulnerabilities can be 
effectively managed and mitigated (while limiting the damage cause by 
false positives). Note: same WAFs have the functionality to replace 
cookies with their own (WAF controlled value)

      e) Why don't they use their WAFs to protect the WAF's web 
interface? Clearly a great test for the usability of a WAF product is to 
use it to protect the complex GUIs used on the WAF management and 
monitoring. Also, when (not if) vulnerabilities are found in their 
product, they could use their WAF to mitigate those security issues.

      g) And what about their website? Are the vendor's websites 
protected by their WAFs?

      h) To the other companies/products that want to get into the WAF 
market, advice to them is: "Create a WAF that allows a good PenTester to 
mitigate 90%+ of the vulnerabilities that he/she discovered". Basically, 
grab HacmeBank and SiteGenerator and use it to take your product were 
the current WAF vendors don't seem to want to go.

      i) What about the Web Application Firewall Evaluation Criteria 
(http://www.webappsec.org/projects/wafec/)? How do this WAF appliances 
rate to this? I might have missed it, but where is the public disclose 
of this information?

   6) After thinking about what happened for a couple days, and trying 
to come up with an explanation for the issues raised on item 5), I think 
that I came up with an answer. The problem is that the WAF technical 
teams don't have Web Penetration Testing experience. They have a 
reasonable level of understanding on how to attack an web application, 
but they are missing the advanced material (I am also puzzled by the 
lack of use of Web Application Security Scanners (WASS)). Maybe the 
solution is for a WAF company and a WASS company to merge and maximize 
the skills sets of both.

   7) Please take my comments with a pinch of salt :) . It is to late 
at night here and I am just doing a brain dump :)

   8) Moving forward, here are some ideas:

         a) Expand the current Hacme Bank challenge to all WAF vendors 
and see who is able to mitigate what. I already asked the 4 WAF vendors 
that participated on the London event to send me the list of issues 
(from Hacme Bank v2.0) that they product is: 
               * Able to protect with default functionality
               * Able to protect with Customization     
               * Not able to protect

         b) Implement all documented HacmeBank vulnerabilities in 
SiteGenerator (also add different types of navigation)

         c) help to organize other Owasp chapter events around the 
world on the same subject and using the same model (Owasp chapter 
leaders let-me know if you are interested)

         d) Implement the Foundstone Validator .Net tool on HacmeBank 
to show how most of those security issues can be resolved when the WAF 
has access to the application layer

         e) write a white paper about WAFs focusing on where they 
should be used and on where they are not effective

         f) expand the analysis of the Hacme Bank vulnerabilities and 
discuss how they could/should be mitigated at WAF

   9) Finally, I just want to give a warning to the WAF  vendors: "you 
must make your appliance invisible to attackers". By invisible I mean 
that it should not be possible to fingerprint your appliance remotely 
from the Internet. Because if you do allow this to occur, then you 
dramatically increase the risk to your clients of being exploited by a 
vulnerability in your WAF (and remember the Witty worm 
(http://www.newsfactor.com/story.xhtml?story_id=23559 and 
http://news.zdnet.co.uk/internet/security/0,39020375,39149459,00.htm) 
which was a targeted exploit on a security product).

That's it, thanks for reading, and I am looking forward to your comments.

Best regards

Dinis Cruz
Owasp .Net Project
www.owasp.net









-------------------------------------------------------------------------
Sponsored by: Watchfire

The Twelve Most Common Application-level Hack Attacks
Hackers continue to add billions to the cost of doing business online 
despite security executives' efforts to prevent malicious attacks. This 
whitepaper identifies the most common methods of attacks that we have seen, 
and outlines a guideline for developing secure web applications. 
Download this whitepaper today!

https://www.watchfire.com/securearea/whitepapers.aspx?id=701300000007t9r
--------------------------------------------------------------------------