RE: SIM and stuff

["Paul Melson" <pmelson () gmail com>]

-----Original Message-----
Subject: Re: SIM and stuff

> "one bucketing expression for the firewall logs"
> OMG, this is so naive :-) Admittedly, a connection denied/allowed might
for some firewalls, > given some luck and planet alignment, be covered by
one regex. But how about all hundreds of > other messages, such as 'failover
failed', 'memory overflowed', 'VPN connection 
> established', etc. They do not even have the above "6 magic fields."
> Some firewalls count up to 1100 distinct messages...

I agree with what you're saying in that most firewalls have a large index of
possible log messages that they can issue.  I further agree that it's not
trivial to parse these.  However, this is more an issue for event
categorization and not correlation.  All correlation boils down to is taking
multiple log sources and finding commonalities in between events.  I think
you're imagining this to be more complicated and difficult than it actually
is.  


> Well, supposedly a majority of Snort sigs and Nessus checks (given two
major open source 
> [well, not quite, in case of Nessus] security
> projects) are supposeduly written by a relatively small group of people.
It is actually 
> kinda fun to do it! But, writing regex parsers for an ever increasing
number of log messages 
> is not nearly as much fun - if it were fun, it would be done by now :-)
Thus, I doubt that 
> community would do a lot here...

I guess it would all depend on community interest and need for these
functions.  That, and I disagree with you about how fun writing log parsers
is.  I have bucketing scripts for PIX (6.1/faddr and 6.2) and Nessus NBE
files that I've written and continue to use.  As work-related coding goes,
it's fairly fun.  I personally think it's cool to take big unruly log files
and make them into useful data.  Plus, for the kind of money your company
could spend on a big SIM, it could easily get a whole library of parsers for
an open source SIM custom written by a contractor.

My ultimate point is that I agree with Thomas Ptacek that this is not hard
stuff and the only reason that this can't be done as an open source project
is a lack of interest or support.


> BTW, this discussion actually belongs on the loganalysis list
> (http://lists.shmoo.com/mailman/listinfo/loganalysis)

I'm not a subscriber, but feel free to cross-post.

PaulM