Re: Intel X520 and Multi-Queue Snort

[Martin Holste <mcholste () gmail com>]

PF_RING will work independently of the card, though it has extra
optimizations for specific models.  I am using stock Broadcom on a 16
core server with 16 Snorts running, all load-balanced using a the
PF_RING flow clustering which hashes on srcip-dstip.  To do this, you
need to use the customized PF_RING Snort DAQ included with the PF_RING
download tarball.  This has been very effective for us, and mostly
hassle-free, though I did need to write a small batch script to start
and shutdown all of the Snort processes.

One note, if you want to cluster on more than 8 cores, you need to set the line:
#define CLUSTER_LEN       8
to be
#define CLUSTER_LEN       16 (or however many cores you have)
in the file PF_RING/kernel/linux/pf_ring.h.  You need to do this
before you compile the kernel module (obviously).  You can always
unload the module, recompile and modprobe it again if you need to
recompile, but don't forget to recompile the libpfring.so and the daq
shared object.

On Thu, May 12, 2011 at 3:42 PM, Mike Lococo <mikelococo () gmail com> wrote:
> Hi Folks,
>
> I'm just getting started testing an Intel X520 capture card, with the
> goal of using it to perform multi-queue snorting.  I'd like to have 8-12
> snort processes each receiving a fraction of the traffic coming in off
> of the 10G physical interface on the card, with traffic distributed in
> some flow-aware manner like hashing the IP/proto/port values for each
> packet.
>
> I understand that linux has some kind of built-in multi-queue
> technology, but I'm not finding any user-space tools to manipulate or
> configure it.  I'm also finding very little high-level documentation or
> discussion of folks that use the feature for network-monitoring
> applications.  Are the built-in linux features useful for scaling snort
> across multiple-cpu's, or is the feature aimed at a fundamentally
> different use-case?
>
> I also understand that pfring can be used with this card, and that there
> is some reasonable documentation around doing so.  Before I got too far
> into that framework, I wanted to see what (if anything) is possible with
> native-linux features.  Is the general consensus among owners of this
> card that PFRING is the way to go?
>
> Cheers,
> Mike Lococo
>
> ------------------------------------------------------------------------------
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users