RE: Content monitorting in Application Security

["Ofer Shezaf" <Ofer.Shezaf () breach com>]

(Mark: As I think the content is interesting I'm resubmitting without,
hopefully, marketing aroma)

Jeremiah Grossman wrote on January 14th:
> Do you think it might be possible to check a documents data format
> validity without actually parsing it into a data structure? I think
> some XML tools might do something like this currently.

I think you nailed down one of the key issues with real time monitoring
of application security: we have to parse a whole lot of content and
make decisions in real time.

The problems with identifying file types are also relevant to detecting
SQL/PHP/JavaScript in inbound traffic required to detect the related
attacks. 

For example using just simple signatures such as select will generate a
lot of false positives, while full parsing to determine SQL syntax,
which is the ultimate way to detect an injection attempt, is not
feasible due to the complexity and performance impact.

While regular expressions may provide a solution, I found that they
don't easily represent a complex file structure of syntax. On top of
that regular expressions are not very efficient either.

In order to tackle this issue I use higher level signatures that
involve multiple signature matches (many of them are string matches but
not all) and different ways of correlating the matches.

A simple example would be SQL detection: Trying to parse input and
determine that it is valid SQL is quite complex, but on the other hand
"select", "union" and a single quote are all pretty bad signatures that
are prone to false positives and do not necessarily indicate SQL
injection. 

On the other hand, All three of them together are much more indicative
of SQL injection, and if you also add a measurement of their proximity
and order of appearance you get a pretty strong signature.

Another example of such higher level signatures is a change detection
that we implemented. In order to do so we learn for each URL its "type"
by finding out which "signatures" are common to all occurrences of this
URL. If for a specific request the page sent did not have enough of
these signatures we know that the page has changed beyond the normal.
The large number of signatures and their aggregation let us detect
changes without full html parsing.


Ofer Shezaf
CTO, Breach Security

Tel: +972.9.956.0036 ext.212
Cell: +972.54.443.1119
ofers () breach com
http://www.breach.com