Re: OS options to monitor traffic over a 1GiB and 10 GiB

[livio Ricciulli <livio () metaflows com>]

You can also check out 
https://www.metaflows.com/technology/10-gbps-pf_ring-2/

It gives detailed instructions on how to build a reliable PF_RING-based 
10G box on CentOS using the Intel 82599 and compare the relative 
performance of PF_RING in NAPI mode and DNA mode. DNA wins but note that 
if you use DNA, you can only attach up 16 Snort processes using the 
Intel 82599 and attach 1 application to the interface (for example 
snort) but you could not (for example) run both Snort and Ntop at the 
same time. NAPI lets you do that.

The number of rules you run is extremely important in determining how 
much bandwidth you can handle. With our traces we could process 4-6 Gigs 
on a dual X5670 and 4000 to 7000 rules respectively.

Livio.

On 06/29/2012 07:41 AM, Robert Vineyard wrote:
> On 6/29/2012 9:23 AM, Joel Esler wrote:
> > Probably BSD. But I think it's less dependent on the OS, and is more dependent on hardware. When you are talking 
> > about 10 Gig, there's lots of factors that come into play.
> Some hardware options I'd recommend, in decreasing order of cost:
>
> http://www.pcapexpress.com/index.php/products
>
> http://www.endace.com/endace-dag-high-speed-packet-capture-cards.html
>
> http://www.silicom-usa.com/10_Gigabit_Ethernet_Networking_Server_Adapters
>
> http://www.myricom.com/products/network-adapters.html
>
> http://ark.intel.com/products/series/46825
>
>
> To monitor that much traffic reliably, you're going to have to employ a load-balancing technique. The best way I've 
> found to go about doing that is to use something that can perform a hash function on the 5-tuple of any given flow. 
> The 5-tuple is composed of the source and destination IP addresses, ports, and protocol. Hashing in this manner 
> ensures that traffic is distributed roughly evenly, and that bidirectional conversations are preserved and sent to 
> the same sensor engine.
>
> The more expensive products in my list above can do this in hardware, often using FPGA tricks and DMA buffering to 
> dramatically accelerate the process. When you're trying to monitor a fully-saturated link, every CPU cycle counts.
>
> The less expensive products (typically from Intel or Myricom) can do some of it in hardware, but they really shine 
> when you pair them with capture-optimized drivers like PF_RING DNA (http://www.ntop.org/products/pf_ring/dna/) or 
> Myricom Sniffer10G (http://www.myricom.com/support/downloads/sniffer.html).
>
> In any case you'll want a big server with lots of CPU cores and as much RAM as you can afford. If you'll be logging 
> payloads and/or expect heavy alert volumes, you'll also need fast disk, like SSD or a hardware RAID10 array. The idea 
> is to run multiple sensor engines (Snort, for example) and bind each one to one of the load-balanced virtual network 
> interfaces presented by the setup I just described. If your traffic is fairly predictable or you have plenty of 
> headroom on your sensor box, you can use CPU affinity to peg those engines to individual cores (there are ways to do 
> this for the firehose of interrupts coming from the NIC too) to avoid spurious context-switching and buy yourself a 
> few more precious CPU cycles. You'll want to run one sensor process per core.
>
> On the other hand, if your traffic is bursty and unpredictable, you may want to forgo the CPU affinity and let the 
> kernel scheduler do its job. For cases like that, I prefer to run two sensor processes per core (doubling the number 
> of required virtual interfaces on your packet-capture NIC). That way, the chunks are smaller and if one needs to 
> burst up to consume a full CPU core, the kernel scheduler will happily relocate the lesser-utilized processes to 
> other cores.
>
> Happy sniffing! :-)
>
> -- Robert Vineyard
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest Snort news!


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!