Re: Rule works in replay file mode, but not when sniffing

[Pavel Rantorski <fhjull01 () outlook com>]

Thank you, the problem was indeed the low snaplen size combined with VLAN use! After setting snaplen to 1542 (1518 
didn't work even though the pcap packets were this large) and changing MTU on network interface accordingly, the rules 
work as expected.

Thanks to everyone involved and have a nice weekend,
Pavel

> Date: Fri, 12 Jul 2013 22:57:08 +0200
> From: rmkml () yahoo fr
> To: fhjull01 () outlook com
> CC: rmkml () yahoo fr
> Subject: Re: [Snort-users] Rule works in replay file mode, but not when sniffing
>
> Hi Pavel,
>
> Can you check mtu on your eth1 network interface please ?
>
> like this on snort.conf :
>
> config snaplen: 1518
>
> (because you have vlan)
>
> If not work, can you post a snort cmd line + snort.conf + snort verbose output please ?
>
> Regards
> @Rmkml
>
>
> On Fri, 12 Jul 2013, Pavel Rantorski wrote:
>
> > > >  Where was this pcap captured?
> >
> > On the same machine/interface Snort is listening on, using 'tshark -i eth1 -w /tmp/dump.pcap'.
> >
> > > > Are you sure you don't have this 
> > > > (https://www.myricom.com/software/myri10ge/349-when-i-view-traffic-with-tcpdump-why-do-i-see-packets-larger-than-the-mtu.html)
> > > >  going on?  I definitely saw evidence of that
> > in your earlier Snort pcap.  If you use the same options to run Snort both in readback and replay modes and you get 
> > different results, it is very likely due to how the packets are presented to Snort.
> >
> > The first pcap may have been captured before I disabled 'generic-receive-offload'. I didn't realize this before you 
> > mentioned it - that would explain the difference. I have checked the article and noticed
> > that 'tcp-segmentation-offload' was still enabled - I disabled it using ethtool, but nothing changed.
> >
> > Currently, the network card features look like this:
> > root@oink:/tmp# ethtool -k eth1
> > Features for eth1:
> > rx-checksumming: on
> > tx-checksumming: on
> > scatter-gather: off
> > tcp-segmentation-offload: off
> > udp-fragmentation-offload: off
> > generic-segmentation-offload: off
> > generic-receive-offload: off
> > large-receive-offload: off
> > rx-vlan-offload: on
> > tx-vlan-offload: on
> > ntuple-filters: off
> > receive-hashing: off
> >
> > (I've done a new pcap with those settings, if you wish to take a look - http://img.dreamchaser.cz/testdata8.zip)
> >
> > I've tried to disable rx/tx checksumming as well, but nothing changed (so I enabled them again). I cannot disable 
> > rx/tx vlan-offload using ethtool - if you think they may be causing the problem, I'll try to
> > look into other ways of disabling them.
> >
> > Can you think of anything else that would be causing the discrepancy between readback/replay modes?
> >
> > I have modified few network-related kernel parameters (Centos6.4) in /etc/sysctl.conf, if that may be relevant:
> >
> > net.core.netdev_max_backlog = 10000
> > net.core.rmem_default = 16777216
> > net.core.rmem_max = 33554432
> >
> > net.ipv4.tcp_rmem = 1048576 4194304 33554432
> > net.ipv4.tcp_no_metrics_save = 1
> >
> >
> >
> > _______________________________________________________________________________________________________________________________________________________________________________________________________________
> > Date: Fri, 12 Jul 2013 12:55:19 -0400
> > Subject: Re: [Snort-users] Rule works in replay file mode, but not when sniffing
> > From: rcombs () sourcefire com
> > To: fhjull01 () outlook com
> > CC: jesler () sourcefire com; snort-users () lists sourceforge net
> >
> >
> >
> > On Fri, Jul 12, 2013 at 12:22 PM, Pavel Rantorski <fhjull01 () outlook com> wrote:
> >       Thank you for your answers.
> >
> >       When I use rule:
> >
> >       alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:”Test file download”; content:”filename=“; 
> > fast_pattern:only; http_header;)
> >
> > It behaves in a same way - won't fire when sniffing, fires when replayed.
> >
> >
> > You are correct that with --dirty-pig, no alert is generated even on replay with the sample I sent. I trimmed the 
> > sample I sent before, perhaps a bit too much. Larger pcap (hopefully with full session)
> > can be found at:
> >
> > http://img.dreamchaser.cz/testdata7.zip (8MB)
> >
> >
> > Where was this pcap captured?
> >
> >       This sample fires both rules (mine + the one suggested by Joel Esler) even with --dirty-pig, but no results 
> > when sniffed directly.
> >
> >
> > Are you sure you don't have this 
> > (https://www.myricom.com/software/myri10ge/349-when-i-view-traffic-with-tcpdump-why-do-i-see-packets-larger-than-the-mtu.html)
> >  going on?  I definitely saw evidence of that in
> > your earlier Snort pcap.  If you use the same options to run Snort both in readback and replay modes and you get 
> > different results, it is very likely due to how the packets are presented to Snort.
> >
> > _______________________________________________________________________________________________________________________________________________________________________________________________________________
> >       Date: Fri, 12 Jul 2013 11:12:05 -0400
> >       Subject: Re: [Snort-users] Rule works in replay file mode, but not when sniffing
> >       From: rcombs () sourcefire com
> >       To: jesler () sourcefire com
> >       CC: fhjull01 () outlook com; snort-users () lists sourceforge net
> >
> >       I think you have something else going on.  Can you send a full session capture?
> >
> >       The capture you have is alerting in readback at shutdown.  To confirm, --dirty-pig to your command line and 
> > you won't get the alert.
> >
> >       The reason is Content-Length: 9156548 but you only have 4101 bytes in the capture.  Snort is trying to 
> > reassemble more data which never shows up.  You can get the alert to fire even with
> >       --dirty-pig if you change paf_max to something like 3072.
> >
> >       If you want it fire in replay you need at least paf_max worth of the response body.
> >
> >       On Fri, Jul 12, 2013 at 10:10 AM, Joel Esler <jesler () sourcefire com> wrote:
> >             What happens when you do a:
> > alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:”Test file download”; content:”filename=“; 
> > fast_pattern:only; http_header;)
> >
> >
> > On Jul 12, 2013, at 9:05 AM, Pavel Rantorski <fhjull01 () outlook com> wrote:
> >
> >       Hello,
> >       I'm testing a rule that should (eventually) detect download/upload of specific file types from public HTTP 
> > servers. I could not get the rule to trigger, so I simplified it to:
> >
> >       alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"Test file download"; 
> > content:"Content-Disposition|3a|"; nocase; http_header; pcre:"/filename=/simH";
> >       classtype:policy-violation; sid:1000004; rev:7;)
> >
> >       (the rule is nowhere near complete, it is simplified to be less prone to mistakes)
> >
> >       Unfortunatelly, the rule still does not work. I captured the traffic (on the same machine/interface that 
> > Snort was running) and verified that such packet is indeed there. When I let
> >       Snort analyze the traffic from this pcap file ('snort -A console -c /etc/snort/snort.conf -r 
> > /tmp/testdata5.pcap -l . -u snort'), the rule is fired on console correctly.
> >
> >       The rule is (in standard, sniffing mode) sometimes triggered as well (although never from this particular 
> > server I am testing).
> >
> >       What could be the cause of this? Snort is running in IDS mode (not inline) and is not dropping packets. LRO 
> > and GRO are disabled on network adapter. I have tried running Snort with
> >       '-k none' without any results.
> >
> >       I have attached small pcap sample of the traffic I'm trying to catch - this is enough to trigger the rule in 
> > replay mode, but didn't trigger when sniffing.
> >
> >       Thank you,
> >       Pavel
> > <testdata5.pcap>------------------------------------------------------------------------------
> > See everything from the browser to the database with AppDynamics
> > Get end-to-end visibility with application monitoring from AppDynamics
> > Isolate bottlenecks and diagnose root cause in seconds.
> > Start your free trial of AppDynamics Pro today!
> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk_______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest Snort news!
> >
> >
> >
> > ------------------------------------------------------------------------------
> > See everything from the browser to the database with AppDynamics
> > Get end-to-end visibility with application monitoring from AppDynamics
> > Isolate bottlenecks and diagnose root cause in seconds.
> > Start your free trial of AppDynamics Pro today!
> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Snort-users mailing list
> > Snort-users () lists sourceforge net
> > Go to this URL to change user options or unsubscribe:
> > https://lists.sourceforge.net/lists/listinfo/snort-users
> > Snort-users list archive:
> > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> >
> > Please visit http://blog.snort.org to stay current on all the latest Snort news!

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk

_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!