Re: Odd bug when using alert_json plugin

[Noah Dietrich <noah_dietrich () 86penny org>]

Ok, i think I've figured out what's going on with this issue, but I'm not
sure it's a good idea the way things are working.

What seems to happen is when you start Snort3, it begins filling the
alert_json.txt file.  Once that file hits the limit (say 100 MB), Snort
re-names this file to alert_json.txt.nnnnnnnnnn, where the n's represent
the unixtime of when the file was re-named.  Snort then creates a new
alert_json.txt file (without the unixtime), and continues to write new
events to that file.

The issues i see with this method of rollover are that log-monitoring
programs won't be expecting file names to change.  So i'd be forced to only
consume only log files with the unixtime portion, meaning i wouldn't get to
consume new events until the log rolled over. If i consumed events from the
original file, there is a good chance that i'd also re-consume (duplicate)
those events again when the filename was renamed.

My request is that when snort starts, it just outputs it's events to a new
file, with the unixtime appended to the filename(
alert_json.txt.nnnnnnnnnn) , and not output to  alert_json.txt at any time,
and especially not rename files. This will ensure that there are no issues
ingesting this data into SIEMs like Splunk or the ELK stack.

Thank you,
Noah





On Wed, Jun 13, 2018 at 9:20 AM, Noah Dietrich <noah_dietrich () 86penny org>
wrote:

> I'm playing around with the alert_json plugin, and i found a bit of odd
> behavior.
>
> When running snort in alert mode, it correctly created the
> /var/log/snort/alert_json.txt file and filled it with events (about 17 MB
> of data in that file). I then restarted snort and pushed a huge amount of
> data at it (sudo ping -c 10000 -i .001 -s 4046 10.0.0.106), and snort
> created a new file with the timestamp (alert_json.txt.1528868120), even
> though the first file hadn't filled up and filled it with 100 MB of data.
>
> I then re-ran snort, with a smaller dataset from local pcaps, and it
> appended the events to the original alert_json.txt file.
>
> I'd call this an odd bug because most applications that consume the json
> files will ignore the first file once the second file is created. I'd say
> there are two issues here:
> 1. Snort rolled over to the second json file early, before the original
> one (without the timestamp) was full. I'm not sure if this was because of
> the large ammount of data i threw at it with the ping command above.
> 2. snort appended alert data to the original alert_json file because it
> wasn't full, even though there was a newer (by UTC timestamp) json file in
> the directory.
>
> i'd recommend that snort not create an initial alert_json file without a
> timestamp if no alert_json files exist, just so there are no differences
> between the output files.
>
> I am running snort with community rules installed (unmodified), two of my
> own local rules (one that alerts on icmp traffic), and builtin rules
> enabled.
>
> First run:
> sudo snort -c /usr/local/etc/snort/snort.lua --pcap-filter \*.pcap
> --pcap-dir ~/snort_src/Pcaps -R /usr/local/etc/snort/rules/snort3-community.rules
> -A alert_json -s 65535 -k none -l /var/log/snort
>
> Second run:
> sudo snort -c /usr/local/etc/snort/snort.lua -i ens160 -l /var/log/snort
> -A alert_json  -R /usr/local/etc/snort/rules/snort3-community.rules
> --warn-all
>
> third run:
> sudo snort -c /usr/local/etc/snort/snort.lua -r ~/maccdc2012_00000.pcap -R
> /usr/local/etc/snort/rules/snort3-community.rules -A alert_json -s 65535
> -k none -l /var/log/snort
> (note: pcap from https://download.netresec.com/pcap/maccdc-2012/
> maccdc2012_00000.pcap.gz)
_______________________________________________
Snort-devel mailing list
Snort-devel () lists snort org
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!