Snort mailing list archives
Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3
From: Victor Roemer <vroemer () sourcefire com>
Date: Wed, 5 Jun 2013 13:08:53 -0400
Martinez, as Joel already mentioned, we'll want to see your Snort configuration. Shutdown stats would also be useful, but perfmon data would be better; if those can be provided. You mentioned that OpenBSD configured the network sysctl parameters "on the fly"; could you direct us to some documentation about this? You also mentioned that Snort was listening on em3, however the startup information in your email indicates that Snort is listening on em4, could you elaborate on this setup? Regarding Suricata, I personally do not have any experience in deploying or configuring it. That said, are you using, relatively, the same configurations? (e.g., any rules enabled, acquiring packets via libpcap, etc..) Also, why are "tcp.reassembly_gap" and "tcp.invalid_checksum" relevant? On Wed, Jun 5, 2013 at 11:06 AM, Joel Esler <jesler () sourcefire com> wrote:
Can you post your snort.conf somewhere? On May 31, 2013, at 2:51 AM, C. L. Martinez <carlopmart () gmail com> wrote:On Thu, May 30, 2013 at 12:45 PM, C. L. Martinez <carlopmart () gmail com>wrote:Hi all, According to the following stats: May 30 11:46:22 nsm01 snort[30096]:===============================================================================May 30 11:46:22 nsm01 snort[30096]: Packet Performance Summary: May 30 11:46:22 nsm01 snort[30096]: max packet time : 10000usecsMay 30 11:46:22 nsm01 snort[30096]: packet events : 654 May 30 11:46:22 nsm01 snort[30096]: avg pkt time : 27.1384usecsMay 30 11:46:22 nsm01 snort[30096]: Rule Performance Summary: May 30 11:46:22 nsm01 snort[30096]: max rule time : 4096usecsMay 30 11:46:22 nsm01 snort[30096]: rule events : 20 May 30 11:46:22 nsm01 snort[30096]: avg rule time : 1.046usecsMay 30 11:46:22 nsm01 snort[30096]:===============================================================================May 30 11:46:22 nsm01 snort[30096]: Packet I/O Totals: May 30 11:46:22 nsm01 snort[30096]: Received: 69971576 May 30 11:46:22 nsm01 snort[30096]: Analyzed: 22427618 ( 32.052%) May 30 11:46:22 nsm01 snort[30096]: Dropped: 41532168 ( 37.247%) May 30 11:46:22 nsm01 snort[30096]: Filtered: 0 ( 0.000%) May 30 11:46:22 nsm01 snort[30096]: Outstanding: 47543958 ( 67.948%) May 30 11:46:22 nsm01 snort[30096]: Injected: 0 May 30 11:46:22 nsm01 snort[30096]:===============================================================================May 30 11:46:22 nsm01 snort[30096]: Breakdown by protocol (includes rebuilt packets): May 30 11:46:22 nsm01 snort[30096]: Eth: 22436767 (100.000%) May 30 11:46:22 nsm01 snort[30096]: VLAN: 0 ( 0.000%) May 30 11:46:22 nsm01 snort[30096]: IP4: 22436767 (100.000%) May 30 11:46:22 nsm01 snort[30096]: Frag: 12 ( 0.000%) May 30 11:46:22 nsm01 snort[30096]: ICMP: 110634 ( 0.493%) May 30 11:46:22 nsm01 snort[30096]: UDP: 752816 ( 3.355%) May 30 11:46:22 nsm01 snort[30096]: TCP: 19433478 ( 86.614%) using snort under OpenBSD 5.3 doesn't returns good performance. Host is a Intel(R) Xeon(R) CPU E5620 @ 2.40GHz, with 8 GiB RAM and four e1000 interfaces. In this sensor, I only use so_rules: # dynamic library rules # include $SO_RULE_PATH/bad-traffic.rules # include $SO_RULE_PATH/chat.rules include $SO_RULE_PATH/dos.rules include $SO_RULE_PATH/exploit.rules # include $SO_RULE_PATH/icmp.rules # include $SO_RULE_PATH/imap.rules include $SO_RULE_PATH/misc.rules include $SO_RULE_PATH/multimedia.rules include $SO_RULE_PATH/netbios.rules # include $SO_RULE_PATH/nntp.rules include $SO_RULE_PATH/p2p.rules include $SO_RULE_PATH/smtp.rules # include $SO_RULE_PATH/snmp.rules include $SO_RULE_PATH/specific-threats.rules include $SO_RULE_PATH/web-activex.rules include $SO_RULE_PATH/web-client.rules include $SO_RULE_PATH/web-iis.rules include $SO_RULE_PATH/web-misc.rules and monitored network is a 1GiB network. Any ideas why??More info: top: load averages: 0.69, 0.65, 0.53 31 processes: 30 idle, 1 on processor CPU0 states: 2.8% user, 0.0% nice, 0.4% system, 20.4% interrupt,76.4% idleCPU1 states: 2.2% user, 0.0% nice, 0.8% system, 0.0% interrupt,97.0% idleCPU2 states: 3.0% user, 0.0% nice, 3.4% system, 0.0% interrupt,93.6% idleCPU3 states: 6.0% user, 0.0% nice, 5.0% system, 0.0% interrupt,89.0% idleMemory: Real: 587M/2947M act/tot Free: 5012M Cache: 2213M Swap: 0K/6142M PID USERNAME PRI NICE SIZE RES STATE WAIT TIME CPUCOMMAND14655 root 4 0 393M 183M sleep/1 bpf 8:44 14.26% snort 25669 root 4 0 1132K 1740K sleep/2 bpf 0:06 3.52%daemonloggersystat ifstat (snort process is listening in em3) 3 users Load 0.89 0.71 0.56 Fri May 3106:23:13 2013IFACE STATE DESC IPKTS IBYTES IERRS OPKTS OBYTES OERRS COLLS em0 up 2 132 0 0 261 0 0 em1 up 0 126 0 0 131 0 0 em2 up 10348 3425952 0 0 0 0 0 em3 up 10346 3425044 0 0 0 0 0 systat mbufs IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System 0 256 185 56 2k 171 435 lo0 em0 2k 6 4 256 6 em1 2k 6 4 256 4 em2 2k 66 4 256 66 em3 2k 65 4 256 65 Stats with ALL so_rules disabled (5 min, more or less): Rule application order: activation->dynamic->pass->drop->sdrop->reject->alert->log Verifying Preprocessor Configurations! ICMP tracking disabled, no ICMP sessions allocated IP tracking disabled, no IP sessions allocated 0 out of 1024 flowbits in use. Packet Performance Monitor Config: ticks per usec : 2417 ticks max packet time : 10000 usecs packet action : fastpath-expensive-packets packet logging : log debug-pkts : disabled Rule Performance Monitor Config: ticks per usec : 2417 ticks max rule time : 4096 usecs rule action : suspend-expensive-rules rule threshold : 5 suspend timeout : 10 secs rule logging : log pcap DAQ configured to passive. Acquiring network traffic from "em4". Reload thread starting... Reload thread started, thread 0xc100dbb8f00 (18056) Decoding Ethernet --== Initialization Complete ==-- ,,_ -*> Snort! <*- o" )~ Version 2.9.4.6 GRE (Build 73) '''' By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team Copyright (C) 1998-2013 Sourcefire, Inc., et al. Using libpcap version 1.3.0 Using PCRE version: 8.31 2012-07-06 Using ZLIB version: 1.2.3 Rules Engine: SF_SNORT_DETECTION_ENGINE Version 1.17 <Build18>Rules Object: web-misc Version 1.0 <Build 1> Rules Object: web-iis Version 1.0 <Build 1> Rules Object: web-client Version 1.0 <Build 1> Rules Object: web-activex Version 1.0 <Build 1> Rules Object: specific-threats Version 1.0 <Build 1> Rules Object: snmp Version 1.0 <Build 1> Rules Object: smtp Version 1.0 <Build 1> Rules Object: p2p Version 1.0 <Build 1> Rules Object: nntp Version 1.0 <Build 1> Rules Object: netbios Version 1.0 <Build 1> Rules Object: multimedia Version 1.0 <Build 1> Rules Object: misc Version 1.0 <Build 1> Rules Object: imap Version 1.0 <Build 1> Rules Object: icmp Version 1.0 <Build 1> Rules Object: exploit Version 1.0 <Build 1> Rules Object: dos Version 1.0 <Build 1> Rules Object: chat Version 1.0 <Build 1> Rules Object: bad-traffic Version 1.0 <Build 1> Preprocessor Object: SF_DNP3 Version 1.1 <Build 1> Preprocessor Object: SF_MODBUS Version 1.1 <Build 1> Preprocessor Object: SF_GTP Version 1.1 <Build 1> Preprocessor Object: SF_REPUTATION Version 1.1 <Build 1> Preprocessor Object: SF_SIP Version 1.1 <Build 1> Preprocessor Object: SF_SDF Version 1.1 <Build 1> Preprocessor Object: SF_DCERPC2 Version 1.0 <Build 3> Preprocessor Object: SF_SSLPP Version 1.1 <Build 4> Preprocessor Object: SF_DNS Version 1.1 <Build 4> Preprocessor Object: SF_SSH Version 1.1 <Build 3> Preprocessor Object: SF_SMTP Version 1.1 <Build 9> Preprocessor Object: SF_IMAP Version 1.0 <Build 1> Preprocessor Object: SF_POP Version 1.0 <Build 1> Preprocessor Object: SF_FTPTELNET Version 1.2 <Build 13> Commencing packet processing (pid=18056) ^C*** Caught Int-Signal===============================================================================Run time for packet processing was 421.51287 seconds Snort processed 630885 packets. Snort ran for 0 days 0 hours 7 minutes 1 seconds Pkts/min: 90126 Pkts/sec: 1498===============================================================================Packet Performance Summary: max packet time : 10000 usecs packet events : 0 avg pkt time : 5.9247 usecs Rule Performance Summary: max rule time : 4096 usecs rule events : 0===============================================================================Packet I/O Totals: Received: 1863847 Analyzed: 630885 ( 33.849%) Dropped: 601452 ( 24.397%) Filtered: 0 ( 0.000%) Outstanding: 1232962 ( 66.151%) Injected: 0===============================================================================Not really good numbers .... Stats with only misc.rules and multimedia.rules (5 min, more or less): Rule application order: activation->dynamic->pass->drop->sdrop->reject->alert->log Verifying Preprocessor Configurations! ICMP tracking disabled, no ICMP sessions allocated IP tracking disabled, no IP sessions allocated WARNING: flowbits key 'file.vqf' is checked but not ever set. WARNING: flowbits key 'file.wmp_playlist' is checked but not ever set. 8 out of 1024 flowbits in use. [ Port Based Pattern Matching Memory ] +- [ Aho-Corasick Summary ] ------------------------------------- | Storage Format : Full-Q | Finite Automaton : DFA | Alphabet Size : 256 Chars | Sizeof State : Variable (1,2,4 bytes) | Instances : 27 | 1 byte states : 26 | 2 byte states : 1 | 4 byte states : 0 | Characters : 1562 | States : 1446 | Transitions : 16926 | State Density : 4.6% | Patterns : 90 | Match States : 88 | Memory (KB) : 562.24 | Pattern : 10.08 | Match Lists : 19.52 | DFA | 1 byte states : 261.06 | 2 byte states : 225.67 | 4 byte states : 0.00 +---------------------------------------------------------------- [ Number of patterns truncated to 20 bytes: 4 ] Packet Performance Monitor Config: ticks per usec : 2422 ticks max packet time : 10000 usecs packet action : fastpath-expensive-packets packet logging : log debug-pkts : disabled Rule Performance Monitor Config: ticks per usec : 2422 ticks max rule time : 4096 usecs rule action : suspend-expensive-rules rule threshold : 5 suspend timeout : 10 secs rule logging : log pcap DAQ configured to passive. Acquiring network traffic from "em4". Reload thread starting... Reload thread started, thread 0x4aa997dc00 (32237) Decoding Ethernet --== Initialization Complete ==-- ,,_ -*> Snort! <*- o" )~ Version 2.9.4.6 GRE (Build 73) '''' By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team Copyright (C) 1998-2013 Sourcefire, Inc., et al. Using libpcap version 1.3.0 Using PCRE version: 8.31 2012-07-06 Using ZLIB version: 1.2.3 Rules Engine: SF_SNORT_DETECTION_ENGINE Version 1.17 <Build18>Rules Object: web-misc Version 1.0 <Build 1> Rules Object: web-iis Version 1.0 <Build 1> Rules Object: web-client Version 1.0 <Build 1> Rules Object: web-activex Version 1.0 <Build 1> Rules Object: specific-threats Version 1.0 <Build 1> Rules Object: snmp Version 1.0 <Build 1> Rules Object: smtp Version 1.0 <Build 1> Rules Object: p2p Version 1.0 <Build 1> Rules Object: nntp Version 1.0 <Build 1> Rules Object: netbios Version 1.0 <Build 1> Rules Object: multimedia Version 1.0 <Build 1> Rules Object: misc Version 1.0 <Build 1> Rules Object: imap Version 1.0 <Build 1> Rules Object: icmp Version 1.0 <Build 1> Rules Object: exploit Version 1.0 <Build 1> Rules Object: dos Version 1.0 <Build 1> Rules Object: chat Version 1.0 <Build 1> Rules Object: bad-traffic Version 1.0 <Build 1> Preprocessor Object: SF_DNP3 Version 1.1 <Build 1> Preprocessor Object: SF_MODBUS Version 1.1 <Build 1> Preprocessor Object: SF_GTP Version 1.1 <Build 1> Preprocessor Object: SF_REPUTATION Version 1.1 <Build 1> Preprocessor Object: SF_SIP Version 1.1 <Build 1> Preprocessor Object: SF_SDF Version 1.1 <Build 1> Preprocessor Object: SF_DCERPC2 Version 1.0 <Build 3> Preprocessor Object: SF_SSLPP Version 1.1 <Build 4> Preprocessor Object: SF_DNS Version 1.1 <Build 4> Preprocessor Object: SF_SSH Version 1.1 <Build 3> Preprocessor Object: SF_SMTP Version 1.1 <Build 9> Preprocessor Object: SF_IMAP Version 1.0 <Build 1> Preprocessor Object: SF_POP Version 1.0 <Build 1> Preprocessor Object: SF_FTPTELNET Version 1.2 <Build 13> Commencing packet processing (pid=32237) ^C*** Caught Int-Signal===============================================================================Run time for packet processing was 368.552024 seconds Snort processed 643495 packets. Snort ran for 0 days 0 hours 6 minutes 8 seconds Pkts/min: 107249 Pkts/sec: 1748===============================================================================Packet Performance Summary: max packet time : 10000 usecs packet events : 0 avg pkt time : 8.95128 usecs Rule Performance Summary: max rule time : 4096 usecs rule events : 0 avg rule time : 1.96408 usecs===============================================================================Packet I/O Totals: Received: 2121798 Analyzed: 643495 ( 30.328%) Dropped: 618918 ( 22.582%) Filtered: 0 ( 0.000%) Outstanding: 1478303 ( 69.672%) Injected: 0===============================================================================About tunning sysctl options, if I am not wrong, OpenBSD tunes them "on the fly" according to network load. And more info: I have installed suricata in this host also to do more tests, and suricata returns me best perfomance without losing many packets: ------------------------------------------------------------------- Counter | TM Name | Value ------------------------------------------------------------------- capture.kernel_packets | RxPcapem51 | 3052575199 capture.kernel_drops | RxPcapem51 | 143259 capture.kernel_ifdrops | RxPcapem51 | 0 decoder.pkts | RxPcapem51 | 19561319 decoder.bytes | RxPcapem51 | 15561225326 decoder.ipv4 | RxPcapem51 | 19561319 decoder.ipv6 | RxPcapem51 | 0 decoder.ethernet | RxPcapem51 | 19561319 decoder.raw | RxPcapem51 | 0 decoder.sll | RxPcapem51 | 0 decoder.tcp | RxPcapem51 | 19561139 decoder.udp | RxPcapem51 | 0 decoder.sctp | RxPcapem51 | 0 decoder.icmpv4 | RxPcapem51 | 180 decoder.icmpv6 | RxPcapem51 | 0 decoder.ppp | RxPcapem51 | 0 decoder.pppoe | RxPcapem51 | 0 decoder.gre | RxPcapem51 | 0 decoder.vlan | RxPcapem51 | 0 decoder.teredo | RxPcapem51 | 0 decoder.ipv4_in_ipv6 | RxPcapem51 | 0 decoder.ipv6_in_ipv6 | RxPcapem51 | 0 decoder.avg_pkt_size | RxPcapem51 | 796 decoder.max_pkt_size | RxPcapem51 | 1506 defrag.ipv4.fragments | RxPcapem51 | 0 defrag.ipv4.reassembled | RxPcapem51 | 0 defrag.ipv4.timeouts | RxPcapem51 | 0 defrag.ipv6.fragments | RxPcapem51 | 0 defrag.ipv6.reassembled | RxPcapem51 | 0 defrag.ipv6.timeouts | RxPcapem51 | 0 defrag.max_frag_hits | RxPcapem51 | 0 tcp.sessions | Detect | 66702 tcp.ssn_memcap_drop | Detect | 0 tcp.pseudo | Detect | 7500 tcp.invalid_checksum | Detect | 2 tcp.no_flow | Detect | 0 tcp.reused_ssn | Detect | 0 tcp.memuse | Detect | 36175872 tcp.syn | Detect | 131466 tcp.synack | Detect | 129929 tcp.rst | Detect | 56046 tcp.segment_memcap_drop | Detect | 0 tcp.stream_depth_reached | Detect | 306 tcp.reassembly_memuse | Detect | 69060696 tcp.reassembly_gap | Detect | 3214 detect.alert | Detect | 38 flow_mgr.closed_pruned | FlowManagerThread | 78944 flow_mgr.new_pruned | FlowManagerThread | 3978 flow_mgr.est_pruned | FlowManagerThread | 2390 flow.memuse | FlowManagerThread | 3852512 flow.spare | FlowManagerThread | 10000 flow.emerg_mode_entered | FlowManagerThread | 0 flow.emerg_mode_over | FlowManagerThread | 0 Relevant data here are tcp.reassembly_gap and tcp.invalid_checksumnumbers.Any idea please??------------------------------------------------------------------------------Get 100% visibility into Java/.NET code with AppDynamics Lite It's a free troubleshooting tool designed for production Get down to code-level detail for bottlenecks, with <2% overhead. Download for free and get started troubleshooting in minutes. http://p.sf.net/sfu/appdyn_d2d_ap2 _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latestSnort news! ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j
_______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users Please visit http://blog.snort.org to stay current on all the latest Snort news!
Current thread:
- Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (May 30)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (May 30)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 Joel Esler (Jun 05)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 Victor Roemer (Jun 05)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 06)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 07)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 12)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 Victor Roemer (Jun 12)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 12)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 Joel Esler (Jun 12)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 13)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 Joel Esler (Jun 05)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (May 30)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 C. L. Martinez (Jun 13)
- Re: Poor performance with Snort 2.9.4.6 under OpenBSD 5.3 waldo kitty (Jun 13)