Snort mailing list archives
Re: Duplicate rules & rule parser
From: Anshuman Anil Deshmukh <anshuman () cybage com>
Date: Fri, 25 Oct 2013 05:15:15 +0000
Hi, I did the changes as suggested by you, but still pulledpork is not able to remove the duplicate rules. How do I remove these duplicate rules? My changed pulledpork.conf as per your suggestion: http://pastebin.com/VYhZFqsC My pulledpork command: sudo perl pulledpork.pl -c /etc/pulledpork070/pulledpork-0.7.0/etc/pulledpork.conf -m /etc/snort/sid-msg.map -I balanced -P -vv Note: I do change the rule state from balanced to security and also sometimes to connectivity as this is my setup setup. This problem exists even before evaluating the rule states. Errors received- WARNING: /etc/snort/preproc_rules/preprocessor.rules(2) GID 105 SID 1 in rule duplicates previous rule. Ignoring old rule. WARNING: /etc/snort/preproc_rules/preprocessor.rules(3) GID 105 SID 2 in rule duplicates previous rule. Ignoring old rule. WARNING: /etc/snort/preproc_rules/preprocessor.rules(4) GID 105 SID 3 in rule duplicates previous rule. Ignoring old rule. WARNING: /etc/snort/preproc_rules/preprocessor.rules(5) GID 105 SID 4 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/chat.rules(2) GID 3 SID 14263 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/dos.rules(5) GID 3 SID 15851 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/dos.rules(8) GID 3 SID 15148 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/dos.rules(17) GID 3 SID 15959 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/dos.rules(20) GID 3 SID 15522 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(2) GID 3 SID 13897 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(3) GID 3 SID 17663 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(8) GID 3 SID 15456 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(9) GID 3 SID 15974 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(10) GID 3 SID 16150 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(11) GID 3 SID 16179 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(12) GID 3 SID 16182 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(14) GID 3 SID 16337 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(16) GID 3 SID 16505 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(22) GID 3 SID 17118 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(23) GID 3 SID 18064 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(24) GID 3 SID 18180 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(33) GID 3 SID 26972 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(37) GID 3 SID 15700 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(41) GID 3 SID 14251 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(42) GID 3 SID 15848 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(43) GID 3 SID 15968 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(46) GID 3 SID 21619 in rule duplicates previous rule. Ignoring old rule. /usr/local/etc/snort/so_rules/exploit.rules(47) GID 3 SID 17741 duplicates previous rule. Using SO rule.. WARNING: /usr/local/etc/snort/so_rules/exploit.rules(48) GID 3 SID 17696 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/multimedia.rules(2) GID 3 SID 17300 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/multimedia.rules(3) GID 3 SID 14252 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/multimedia.rules(4) GID 3 SID 14253 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/multimedia.rules(5) GID 3 SID 14254 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/netbios.rules(2) GID 3 SID 15528 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/netbios.rules(3) GID 3 SID 16395 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/netbios.rules(12) GID 3 SID 16396 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/smtp.rules(4) GID 3 SID 15329 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/snmp.rules(2) GID 3 SID 17632 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(2) GID 3 SID 18213 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(3) GID 3 SID 18400 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(4) GID 3 SID 18405 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(5) GID 3 SID 18409 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(6) GID 3 SID 18410 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(7) GID 3 SID 18411 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(8) GID 3 SID 18412 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(9) GID 3 SID 18421 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(12) GID 3 SID 18501 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(13) GID 3 SID 18502 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(14) GID 3 SID 18504 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/specific-threats.rules(15) GID 3 SID 18505 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(3) GID 3 SID 16370 in rule duplicates previous rule. Ignoring old rule. /usr/local/etc/snort/so_rules/web-client.rules(5) GID 3 SID 13975 duplicates previous rule. Using SO rule.. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(7) GID 3 SID 13977 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(10) GID 3 SID 15503 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(12) GID 3 SID 16156 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(13) GID 3 SID 16158 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(14) GID 3 SID 16228 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(15) GID 3 SID 16320 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(16) GID 3 SID 16658 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(18) GID 3 SID 17199 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(19) GID 3 SID 17201 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(30) GID 3 SID 17765 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(31) GID 3 SID 17762 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(32) GID 3 SID 18063 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(34) GID 3 SID 18673 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(35) GID 3 SID 21352 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(38) GID 3 SID 13879 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(46) GID 3 SID 13969 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(51) GID 3 SID 13954 in rule duplicates previous rule. Ignoring old rule. /usr/local/etc/snort/so_rules/web-client.rules(52) GID 3 SID 15117 duplicates previous rule. Using SO rule.. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(53) GID 3 SID 14260 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(55) GID 3 SID 15125 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(56) GID 3 SID 15300 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(57) GID 3 SID 15498 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(58) GID 3 SID 15454 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(60) GID 3 SID 15465 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(62) GID 3 SID 15521 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(63) GID 3 SID 15520 in rule duplicates previous rule. Ignoring old rule. /usr/local/etc/snort/so_rules/web-client.rules(65) GID 3 SID 15857 duplicates previous rule. Using SO rule.. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(66) GID 3 SID 16230 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(67) GID 3 SID 15920 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(68) GID 3 SID 17242 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(69) GID 3 SID 16415 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(70) GID 3 SID 16530 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(71) GID 3 SID 14772 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(72) GID 3 SID 16343 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(73) GID 3 SID 23180 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(74) GID 3 SID 17665 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(75) GID 3 SID 15975 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(76) GID 3 SID 15976 in rule duplicates previous rule. Ignoring old rule. /usr/local/etc/snort/so_rules/web-client.rules(78) GID 3 SID 17700 duplicates previous rule. Using SO rule.. WARNING: /usr/local/etc/snort/so_rules/web-client.rules(80) GID 3 SID 15433 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-misc.rules(3) GID 3 SID 15470 in rule duplicates previous rule. Ignoring old rule. WARNING: /usr/local/etc/snort/so_rules/web-misc.rules(7) GID 3 SID 23182 in rule duplicates previous rule. Ignoring old rule. Thanks and Regards, Anshuman -----Original Message----- From: Anshuman Anil Deshmukh [mailto:anshuman () cybage com] Sent: Thursday, October 24, 2013 8:03 AM To: snort-users () lists sourceforge net Subject: Re: [Snort-users] Duplicate rules & rule parser Also request you to see my comments/questions inline for the oddities and let me know your views on it. Thanks and Regards, Anshuman Regards, Anshuman Anil Deshmukh // Information Security-Analyst Phone: 91-20-66041700, 91-20-66044700 (Extn. 6114) Cell: 91-99230-51641 -----Original Message----- From: JJ Cummings [mailto:cummingsj () gmail com] Sent: Thursday, October 24, 2013 7:47 AM To: Anshuman Anil Deshmukh Cc: snort-users () lists sourceforge net Subject: Re: [Snort-users] Duplicate rules & rule parser Pulledpork is a working parser, when used correctly. Sent from the iRoad
On Oct 23, 2013, at 12:41, Anshuman Anil Deshmukh
<anshuman () cybage com>
wrote:
Hi,
I am still hunting for a rule parser, does anyone have a working rule
parser? Also request you to see my comments/questions inline for the oddities.
Thanks and Regards,
Anshuman
-----Original Message-----
From: Peter Bates [mailto:peter.bates () ucl ac uk]
Sent: Tuesday, October 22, 2013 7:22 PM
To: snort-users () lists sourceforge net
Subject: Re: [Snort-users] Duplicate rules & rule parser
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello all
On 22/10/2013 14:14, Anshuman Anil Deshmukh wrote:
Since last week I am observing that I have duplicate copies of many
rules
in my rule file. I am using pulledpork version 0.70 and hence all the rules are merged into a single file. How do we remove all the duplicate rules? Any rule parser that can be used or anybody having a script for that?
My pulledpork configuration - http://pastebin.com/E4Z368rx
Oddities I can see:
1) You've defined a reg-rules rule_url and the community-rules.
[Anshuman] Yes, I am a registered user. So in that case do you mean to
say
that both community & registered user rules doesn't work together and hence rather than using both of them, either one of them should be used?
2) You've put a fixed Snort version (2.9.5.0) into the reg-rules
rule_url [Anshuman] I request you to kindly send me the correct URL
meant for registered users
3) You've set snort_version to 2.9.5.0 - as long as Pulledpork can
find
the binary, you should let it work it out for itself.
[Anshuman] No further queries on this
- --
Peter Bates
Senior Information Security Officer Phone: +44(0)2076792049
Information Services Division Internal Ext: 32049
University College London
London WC1E 6BT
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.13 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJSZoLiAAoJELhVoVpEMS6RxqYIAKIv3/KZ4PcJjjRjg0wqoiIs
7tqbHpbl6ecc9QZ9Kg0S1QbMMEDtbplfwYn0rnpDzPL6mUbD93X0AWA4IhfRQFk4
UILG/YuhXwD1eTzkIq4F5Uy0Ln0hPjHnQaPpF5UNVvZi9L830pkRL/RjY9bbL7Qv
Kq6qhh8Jlco2wQ/kCdDbldoT9pKck/mpTsDkm2sfVEZSx2vtKLvj+rkOz0lf7lIr
7WVfQiv9KB1TGs7FxALSjE8HayQCz/tX0+jkKibrd+ds74Kj5TQ3RvA362hHgW+Y
r44k6z/3na9ZXpsu+xfRlQ2N5GK8FlBsbqO3AScKO3dfumnfux5fYQxPNbEPcOc=
=c+y4
-----END PGP SIGNATURE-----
----------------------------------------------------------------------
-------- October Webinars: Code for Performance Free Intel webinars
can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
most from the latest Intel processors and coprocessors. See abstracts
and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.c
lktrk _______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
Please visit http://blog.snort.org to stay current on all the latest
Snort
news!
"Legal Disclaimer: This electronic message and all contents contain
information from Cybage Software Private Limited which may be privileged, confidential, or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is strictly prohibited. If you have received this electronic message in error please notify the sender by reply e-mail to and destroy the original message and all copies. Cybage has taken every reasonable precaution to minimize the risk of malicious content in the mail, but is not liable for any damage you may sustain as a result of any malicious content in this e-mail. You should carry out your own malicious content checks before opening the e-mail or attachment."
www.cybage.com
----------------------------------------------------------------------
-------- October Webinars: Code for Performance Free Intel webinars
can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the
most from the latest Intel processors and coprocessors. See abstracts
and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.c
lktrk _______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
Please visit http://blog.snort.org to stay current on all the latest
Snort
news! "Legal Disclaimer: This electronic message and all contents contain information from Cybage Software Private Limited which may be privileged, confidential, or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is strictly prohibited. If you have received this electronic message in error please notify the sender by reply e-mail to and destroy the original message and all copies. Cybage has taken every reasonable precaution to minimize the risk of malicious content in the mail, but is not liable for any damage you may sustain as a result of any malicious content in this e-mail. You should carry out your own malicious content checks before opening the e-mail or attachment." www.cybage.com "Legal Disclaimer: This electronic message and all contents contain information from Cybage Software Private Limited which may be privileged, confidential, or otherwise protected from disclosure. The information is intended to be for the addressee(s) only. If you are not an addressee, any disclosure, copy, distribution, or use of the contents of this message is strictly prohibited. If you have received this electronic message in error please notify the sender by reply e-mail to and destroy the original message and all copies. Cybage has taken every reasonable precaution to minimize the risk of malicious content in the mail, but is not liable for any damage you may sustain as a result of any malicious content in this e-mail. You should carry out your own malicious content checks before opening the e-mail or attachment." www.cybage.com
------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
_______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
Current thread:
- Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 22)
- Re: Duplicate rules & rule parser JJ Cummings (Oct 22)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 22)
- Re: Duplicate rules & rule parser Joel Esler (Oct 22)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 22)
- Re: Duplicate rules & rule parser Peter Bates (Oct 22)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 22)
- Re: Duplicate rules & rule parser Peter Bates (Oct 22)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 23)
- Re: Duplicate rules & rule parser JJ Cummings (Oct 23)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 23)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 24)
- Re: Duplicate rules & rule parser Joel Esler (Oct 25)
- Re: Duplicate rules & rule parser JJC (Oct 25)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 26)
- Re: Duplicate rules & rule parser Eric G (Oct 26)
- Re: Duplicate rules & rule parser Joel Esler (Oct 26)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 27)
- Re: Duplicate rules & rule parser Anshuman Anil Deshmukh (Oct 22)
- Re: Duplicate rules & rule parser JJ Cummings (Oct 22)
- Re: Duplicate rules & rule parser Peter Bates (Oct 24)