RE: snortreport -- SLOOOW

["John Berkers" <berjo () ozemail com au>]

Jason,

I have a similar sized dataset in one of my snort databases (yes I've got a
couple, most with similar sized datasets).  Using postgres (originally) I
found that on a P Pro 180 with 128Mb RAM & 1+4Gb SCSI (40Mbps) ACID was
painfully slow.  Since switching to MySQL I have had much better response.
Plus, using the chaching feature on Acid 0.9.6b13 improves performance
somewhat as well, esp if you set it to manually cache.

I find that Acid is generally more useful for identifying particular types
of traffic than either snortreport or Demarc, although both have their uses.
I actually use all of them in tandem to analyse my data, except that
Snortreport does not seem to respond at all on my large dataset.

If you're using Demarc you need to create the database using Demarc's create
tables script, then run Acid on the database to get it to create it's
tables.  Snort report doesn't utilise it's own tables, so there's no hassle
there.

I'm slowly working on reducing our false positives, but with a Class couple
(up to 5) class C's and a Class B to protect it is quite a daunting task.

Hope that helps some.

Regards,
John Berkers                                       ICQ: 112912
Network Services              -             Hansen Corporation
john.berkers () hancorp com au               berjo () ozemail com au


-----Original Message-----
From: snort-users-admin () lists sourceforge net
[mailto:snort-users-admin () lists sourceforge net]On Behalf Of Jacob
Killian
Sent: Thursday, 30 August 2001 8:09
To: Jason Costomiris
Cc: snort-users () lists sourceforge net
Subject: Re: [Snort-users] snortreport -- SLOOOW



*** PGP Signature Status: good
*** Signer: Jacob Killian <jacob () pgtc com> (Invalid)
*** Signed: 30/08/2001 8:08:51 AM
*** Verified: 30/08/2001 10:54:37 PM
*** BEGIN PGP VERIFIED MESSAGE ***

Yikes is right.  That's accumulated over a couple of days (yup).

It's an ISP network, so I get all the BS, but I also have ALOT of false
positives...i.e., just about any ICMP traffic gets logged.  I was hoping to
use snortreport to help me in classifying and, thus, reducing the number of
false positives, but that doesn't seem to be the way to go about it.  It
might have to be a manual process.  Maybe I can use snort-stat to help pare
down the false positives?

It currently takes most of my day working with grep to go through the snort
alerts, so I haven't had time to go through the rules, so it takes me most
of
8 hours to grep through the logs, so I haven't had time to go through the
rules, so it takes me most of 8 hours to grep...etc, etc, etc.  One of those
catch-22s.

Is ACID any better at handling large data sets?

Thanks again,
Jacob

On Wednesday 29 August 2001 04:09 pm, Jason Costomiris wrote:
> On Wed, Aug 29, 2001 at 03:00:22PM -0500, Jacob Killian wrote:
> : CPU: 600Mhz AMD Athalon
> : Mem: 384M, w/ 512M Swap
> : Alerts: 257792 records in the event table (  :~ }  << peevish grin.
> : Haven't worked on reducing the number of false positives yet -- get
> : alerts for ICMP traffic, etc.  I was hoping to use snortreport to help
> : with that).
>
> Yikes.  Over what time period did you accumulate that number of alerts?
> Do you have a lot of false positives in that mix?
>
> : While a report is being run, I get an instance of mysqld running with
> : maximum CPU utilization (it does play nice, but will use 97% if nothing
> : else is running).  Memory utilization is fine (doesn't even use any of
> : the swap space).
>
> That's the behavior I see too.
>
> : I guess I need to work on reducing the number of alerts before I work
> : with snortreport anymore?
>
> You might want to consider some sort of db archival process, unless all
> those alerts were generated over a very short time.
>
> : Is there a way to get statistical info from snort
> : (packets processed, packets dropped, alerts triggered, etc)?
>
> I doubt you can get the number of packets processed, since not every
packet
> is being logged (unless you've specifically told it to do so!).  As for
> number of packets dropped, I highly doubt that number's recorded anywhere.
> Number of alerts triggered - that's already done by snortreport.
>
> : Who's working ot the SQL optimization?
>
> Chris Adams said he was going to spend some time doing some optimization
> on the SQL...

--
Jacob Killian
System Administrator
PGTC Internet

jacob () pgtc com
http://www.pgtc.com
501-846-7245

============================================
Such folly friend
Such a waste of time
You could get paid for just such a crime
There's only money
There's only fame
When you play
The vandal's game
And they're gratin' in
The Goodnight holler
To get a piece of the real estate dollar
One million years
Of the master's landscape
Gone with a pen stroke smile and a handshake
Oh me oh my it's gone it's gone it's gone

--"BARREL SPRINGS" (Mark Bilyeu/Jody Bilyeu)
============================================

*** END PGP VERIFIED MESSAGE ***

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
http://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users


_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
http://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users