Snort mailing list archives
Re: Snort++: how to get multithreading to work?
From: Russ <rucombs () cisco com>
Date: Wed, 17 Jun 2015 14:01:25 -0400
My bad ... didn't see all the output you provided before firing off that response.
Currently load balancing must be done externally which means you get one packet thread per source. If you have just one interface, one packet thread is all you get.
On 6/17/15 1:41 PM, Prude, Terrell (SCC) wrote:
Hello folks,This is my first post. We’ve been running “regular” Snort since the 2.9.5.x days and thought we’d give the new Snort 3.0.0 Alpha a whirl. For us, the major attraction to Snort++ is the multithreading for reasons of capacity.Unfortunately, I’m having some trouble figuring out how to get that to work. So far, the Snort process looks like it’s still using only one CPU. Snort itself seems to start right up and is “snorting” packets, and we are getting output in the Unified2 format.Could someone point me in the right direction as to what I’m missing? Platform: ------------------------------------ Processor: Intel 4GHz quad-core w/ hyperthreading DRAM: 32 GB Disk space: 2TB, with about 1.9TB free NIC for Snorting: Intel X520-SR2 10Gbit fiber Ethernet NIC for management: Realtek 8169 built-in 1Gbit copper Ethernet OS: CentOS 7.1 Snort version: 3.0.0-a1-155 LuaJIT version: 2.0.4 DAQ version: 2.0.5 ------------------------------------All the ./configure stuff uses the default paths, i. e. the /usr/local tree. I tried to stay as plain-vanilla as I reasonably could that way.The configure statement: ------------------------------------./configure --disable-silent-rules --enable-ppm --enable-perf-profiling --enable-large-pcap------------------------------------I then ran the make statement with -j8, per the directions, followed by “make install”. That looked good.Command line to invoke Snort: ------------------------------------/usr/local/bin/snort -D -i enp1s0f0 -c /usr/local/etc/snort/snort.lua -l /var/log/snort -z 8------------------------------------ The log output from when Snort starts: ------------------------------------Jun 17 04:07:47 p-its-idssnort2 snort[2984]: --------------------------------------------------Jun 17 04:07:47 p-its-idssnort2 snort[2984]: o")~ Snort++ 3.0.0-a1-155Jun 17 04:07:47 p-its-idssnort2 snort[2984]: --------------------------------------------------Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading /usr/local/etc/snort/snort.lua:Jun 17 04:07:48 p-its-idssnort2 snort[2984]: ips Jun 17 04:07:48 p-its-idssnort2 snort[2984]: active Jun 17 04:07:48 p-its-idssnort2 snort[2984]: classifications Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rpc_decode Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_tcp Jun 17 04:07:48 p-its-idssnort2 snort[2984]: binder Jun 17 04:07:48 p-its-idssnort2 snort[2984]: unified2 Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_ip Jun 17 04:07:48 p-its-idssnort2 snort[2984]: event_queue Jun 17 04:07:48 p-its-idssnort2 snort[2984]: detection Jun 17 04:07:48 p-its-idssnort2 snort[2984]: network Jun 17 04:07:48 p-its-idssnort2 snort[2984]: normalizer Jun 17 04:07:48 p-its-idssnort2 snort[2984]: references Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_udp Jun 17 04:07:48 p-its-idssnort2 snort[2984]: search_engineJun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished /usr/local/etc/snort/snort.lua.Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading rules:Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading /usr/local/etc/snort/rules/local.rules:Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished /usr/local/etc/snort/rules/local.rules.Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished rules.Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule counts Jun 17 04:07:48 p-its-idssnort2 snort[2984]: total rules loaded: 2304 Jun 17 04:07:48 p-its-idssnort2 snort[2984]: text rules: 2304 Jun 17 04:07:48 p-its-idssnort2 snort[2984]: option chains: 2304 Jun 17 04:07:48 p-its-idssnort2 snort[2984]: chain headers: 2304Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule port counts Jun 17 04:07:48 p-its-idssnort2 snort[2984]: tcp udp icmp ipJun 17 04:07:48 p-its-idssnort2 snort[2984]: any 2304 2304 2304 2304Jun 17 04:07:48 p-its-idssnort2 snort[2984]: nc 0 0 0 2304Jun 17 04:07:48 p-its-idssnort2 snort[2984]: --------------------------------------------------Jun 17 04:07:48 p-its-idssnort2 snort[2984]: pcap DAQ configured to passive.Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Initializing daemon modeJun 17 04:07:48 p-its-idssnort2 snort[2993]: Daemon initialized, signaled parent pid: 2984Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Writing PID "2993" to file "/var/log/snort/snort.pid"Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Commencing packet processing Jun 17 04:07:48 p-its-idssnort2 snort[2993]: ++ [0] enp1s0f0 ------------------------------------ The log output after I kill the Snort process: ------------------------------------ Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ** caught term signal Jun 17 05:33:04 p-its-idssnort2 snort[2993]: == stopping Jun 17 05:33:04 p-its-idssnort2 snort[2993]: -- [0] enp1s0f0Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Packet StatisticsJun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: daq Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pcaps: 1 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: received: 1120415147 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: dropped: 1096740235 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: outstanding: 1096741527 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: allow: 23673620 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: idle: 1Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: codecJun 17 05:33:04 p-its-idssnort2 snort[2993]: total: 23673624 (100.000%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: other: 555 ( 0.002%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 2609430 ( 11.023%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: auth: 769 ( 0.003%) Jun 17 05:33:04 p-its-idssnort2 snort[2993]: esp: 211987 ( 0.895%) Jun 17 05:33:04 p-its-idssnort2 snort[2993]: eth: 23673624 (100.000%) Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gre: 8574 ( 0.036%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4: 2671 ( 0.011%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4_ip: 1277 ( 0.005%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp6: 26 ( 0.000%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv4: 23673624 (100.000%) Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6: 58 ( 0.000%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6_no_next: 31 ( 0.000%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ppp_encap: 8574 ( 0.036%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp: 16224198 ( 68.533%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: teredo: 58 ( 0.000%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp: 4838675 ( 20.439%)Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Module StatisticsJun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp Jun 17 05:33:04 p-its-idssnort2 snort[2993]: bad checksum (ip4): 10418Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: binder Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 4399149 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: inspects: 4399149Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ip flows: 2800 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp flows: 4296173 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp prunes: 4165102 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp flows: 100176Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_ip Jun 17 05:33:04 p-its-idssnort2 snort[2993]: fragments: 220 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: reassembled: 4 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers added: 216 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers freed: 216 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes inserted: 220 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes deleted: 220Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_tcp Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 4296173 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 170170 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: events: 3999594 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn trackers: 247856 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn-ack trackers: 6903 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: data trackers: 109476 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers created: 364235 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers released: 364235 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs queued: 335355 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs released: 335355 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs split: 227 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs used: 56291 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt packets: 22755 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt buffers: 42889 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: overlaps: 28 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gaps: 73264 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max segs: 15128 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max bytes: 137382 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: client cleanups: 73351 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: server cleanups: 66854Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_udp Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 100176 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: created: 100176 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: released: 100176Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Summary StatisticsJun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: detection Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: process Jun 17 05:33:04 p-its-idssnort2 snort[2993]: signals: 1Jun 17 05:33:04 p-its-idssnort2 snort[2993]: --------------------------------------------------Jun 17 05:33:04 p-its-idssnort2 snort[2993]: timing Jun 17 05:33:04 p-its-idssnort2 snort[2993]: runtime: 01:25:16 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: seconds: 5116.16403 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 23673620 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pkts/sec: 4627 Jun 17 05:33:04 p-its-idssnort2 snort[2993]: o")~ Snort exiting ------------------------------------ The “top” output while Snort++ is running: ------------------------------------ top - 05:20:58 up 2:15, 3 users, load average: 1.00, 1.01, 0.99Tasks:*201 *total,* 1 *running,*200 *sleeping,* 0 *stopped,* 0 * zombie%Cpu0 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu1 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*99.7 *id,*0.3 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu2 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu3 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu4 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu5 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu6 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * hi,* 0.0 *si,* 0.0 *st%Cpu7 :*97.3 *us,* 0.0 *sy,* 0.0 *ni,* 0.0 *id,*0.0 *wa,* 0.0 * hi,* 2.7 *si,* 0.0 *stKiB Mem :*32703168 *total,*31865964 *free,* 659032 *used,*178172 *buff/cacheKiB Swap:* 4092 *total,* 4092 *free,* 0 *used.*31846588 *avail MemPID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2993 root 20 0 379776 327360 4040 S 100.0 1.0 73:23.35 snort1 root 20 0 56652 6728 3908 S 0.0 0.0 0:00.76 systemd2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd3 root 20 0 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/05 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H7 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/08 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/010 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/111 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/212 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/313 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/414 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/515 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/616 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcuob/717 root 20 0 0 0 0 S 0.0 0.0 0:00.22 rcu_sched18 root 20 0 0 0 0 S 0.0 0.0 0:00.09 rcuos/019 root 20 0 0 0 0 S 0.0 0.0 0:00.08 rcuos/120 root 20 0 0 0 0 S 0.0 0.0 0:00.01 rcuos/221 root 20 0 0 0 0 S 0.0 0.0 0:00.01 rcuos/322 root 20 0 0 0 0 S 0.0 0.0 0:00.03 rcuos/423 root 20 0 0 0 0 S 0.0 0.0 0:00.01 rcuos/524 root 20 0 0 0 0 S 0.0 0.0 0:00.03 rcuos/625 root 20 0 0 0 0 S 0.0 0.0 0:00.01 rcuos/726 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/027 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/128 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/129 root 20 0 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/1------------------------------------And finally, what the NIC itself is reporting for traffic that it’s seeing. We’re seeing it come in, all right. J So far, no errors, collisions, or any other apparent nasties.------------------------------------ $ ip -s link show enp1s0f03: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000link/ether 90:e2:ba:85:28:74 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 1865322070123 1892842032 0 0 0 8445 TX: bytes packets errors dropped carrier collsns 0 0 0 0 0 0 ------------------------------------ ------------------------------------------------------------------------------ _______________________________________________ Snort-devel mailing list Snort-devel () lists sourceforge net https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!
------------------------------------------------------------------------------
_______________________________________________ Snort-devel mailing list Snort-devel () lists sourceforge net https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!
Current thread:
- Snort++: how to get multithreading to work? Prude, Terrell (SCC) (Jun 17)
- Re: Snort++: how to get multithreading to work? Russ (Jun 17)
- Re: Snort++: how to get multithreading to work? Russ (Jun 17)
- Re: Snort++: how to get multithreading to work? elof (Jun 22)