Re: Trigger anomalies (on LXC container versus host)

[Chris <berzerkatives () gmail com>]

Hi Al,

Sorry for taking a while to get back. The reason why my .pcaps weren't
triggering on the hypervisor is because I wasn't capturing enough
packets (I'd limited the capture to local (attacking) machine
destination port 80). Now that I've got all of the communication between
my attacking machine, and the remote server, reading those pcaps into
Snort on the hypervisor does cause an alert.

So now the problem is much clearer.

To recap: running the live attack causes the hypervisor to alert,
however the container does not.

Reading the captured attack from pcaps cause the hypervisor to alert,
even without the "-k none". More interestingly, on the container (using
the data captured insode the container), using "-k none" causes it to
alert! And omitting "-k none" results in no triggering.

This suggests to me that the iptables NAT'ing is screwing with the
checksums, is that expected and normal? and if so shall I just
configure Snort to ignore checksums?

Thanks for your help,
Chris

On Mon, 4 May 2015 01:03:17 +0000
"Al Lewis (allewi)" <allewi () cisco com> wrote:

> Try running it with the "-k none" flag set (to ignore checksums).
>
> Albert Lewis
> QA Software Engineer
> SOURCEfire, Inc. now part of Cisco
> 9780 Patuxent Woods Drive
> Columbia, MD 21046 
> Phone: (office) 443.430.7112
> Email: allewi () cisco com 
>
>
> -----Original Message-----
> From: Al Lewis (allewi) 
> Sent: Sunday, May 03, 2015 8:51 PM
> To: Chris
> Cc: snort-users () lists sourceforge net
> Subject: Re: [Snort-users] Trigger anomalies (on LXC container versus
> host)
>
> Its pretty hard to say without the pcap and conf file.
>
>
> Albert Lewis
> QA Software Engineer
> SOURCEfire, Inc. now part of Cisco
> 9780 Patuxent Woods Drive
> Columbia, MD 21046
> Phone: (office) 443.430.7112
> Email: allewi () cisco com 
>
> -----Original Message-----
> From: Chris [mailto:berzerkatives () gmail com]
> Sent: Sunday, May 03, 2015 8:08 PM
> To: Al Lewis (allewi)
> Cc: snort-users () lists sourceforge net
> Subject: Re: [Snort-users] Trigger anomalies (on LXC container versus
> host)
>
> Hi Al,
>
> I've taken the pcap capture on the hypervisor (which is where it will
> trigger), and run it through Snort using -r, and I see that the
> correct number of packets are being read from the capture, and that
> one of them is processed (I guess for its for HTTP payload, whereas
> the other packets I believe are just initialising the TCP
> connection), however it doesn't cause it to trigger.
>
> I am certain that the packet that triggers the alert is in here, as
> looking through the earlier Snort alert lot I see this.
>
> [**] [1:1646:5] WEB-CGI test.cgi access [**]
> [Classification: access to a potentially vulnerable web application]
> [Priority: 2] 05/03-22:54:17.553955 XXX.XXX.XXX.XXX:55208 ->
> YYY.YYY.YYY.YYY:80 TCP TTL:64 TOS:0x0 ID:33526 IpLen:20 DgmLen:137 DF
> ***AP*** Seq: 0x934BB66C  Ack: 0xF99F9563  Win: 0x3890  TcpLen: 32
>
> And in the pcap capture I have an identically timestamped and
> addressed packet.
>
> 22:54:17.553955 IP XXX.XXX.XXX.XXX.55208 > YYY.YYY.YYY.YYY.80: Flags
> [P.], seq 0:85, ack 1, win 229, options [nop,nop,TS val 9149570 ecr
> 785127305], length 85
>
> But loading it up, with identical command line arguments in addition
> to the -r, minus the -D, and having it log to a brand new place,
> fails to log/trigger anything.
>
> Here's the rule that one would expect to trigger.
>
> alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS
> (msg:"WEB-CGI test.cgi access"; flow:to_server,established;
> uricontent:"/test.cgi"; nocase; classtype:web-application-activity;
> sid:1646; rev:5;)
>
> I'm left to think that I must be making a mistake somehow with my
> playback testing, but I'm not sure what. Any ideas?
>
>
> On Sun, 3 May 2015 23:03:55 +0000
> "Al Lewis (allewi)" <allewi () cisco com> wrote:
>
> > The pcaps are needed for replay and testing against. 
> >
> > As a test... if you run snort from both of your other instances and 
> > replay the packets  the outputs should be the same when using the
> > "-r" flag.
> >
> > If the results are the same then you know you have a problem with
> > your setup and with packets coming off the wire.
> >
> > You could also replay the pcaps (using tcpreplay ) and look at the 
> > exit stats to see if the packets are making it into snort correctly.
> >
> >
> > Albert Lewis
> > QA Software Engineer
> > SOURCEfire, Inc. now part of Cisco
> > 9780 Patuxent Woods Drive
> > Columbia, MD 21046
> > Phone: (office) 443.430.7112
> > Email: allewi () cisco com
> >
> > -----Original Message-----
> > From: Chris [mailto:berzerkatives () gmail com]
> > Sent: Sunday, May 03, 2015 6:33 PM
> > To: Al Lewis (allewi)
> > Cc: snort-users () lists sourceforge net
> > Subject: Re: [Snort-users] Trigger anomalies (on LXC container
> > versus host)
> >
> > Hi Albert,
> >
> > Absolutely, thanks for getting back to me, and I'd be more than
> > happy to provide extra information.
> >
> > I'm guessing pcaps wouldn't be of any use as running tcpdump on the 
> > container and hypervisor at the same time yielded the exact same 
> > packets (that one system flagged, and the other ignored). I've just 
> > taken a moment to diff the packet capture that would be expected to 
> > trigger Snort, and the only difference is a very slight timestamp 
> > difference.
> >
> > Like I say, the configs are pretty much vanilla Debian with the 
> > smallest amount of tweaking for interface names, and not much else.
> > Should I just tar up /etc/snort and send it over?
> >
> >
> >
> > On Sun, 3 May 2015 19:04:27 +0000
> > "Al Lewis (allewi)" <allewi () cisco com> wrote:
> >
> > > Hello,
> > >
> > >   It would help if you could provide some pcaps of the
> > > traffic in question. Also a snort.conf or the rules that are
> > > involved.
> > >
> > > Thanks!
> > >
> > > Albert Lewis
> > > QA Software Engineer
> > > SOURCEfire, Inc. now part of Cisco
> > > 9780 Patuxent Woods Drive
> > > Columbia, MD 21046
> > > Phone: (office) 443.430.7112
> > > Email: allewi () cisco com
> > >
> > > -----Original Message-----
> > > From: Chris [mailto:berzerkatives () gmail com]
> > > Sent: Sunday, May 03, 2015 9:31 AM
> > > To: snort-users () lists sourceforge net
> > > Subject: [Snort-users] Trigger anomalies (on LXC container versus
> > > host)
> > >
> > > I'm observing a problematic difference in behaviour between two 
> > > instances of Snort that are configured identically (recursive 
> > > diff'ed their config dirs, and compared their initialisation
> > > outputs) aside from the required differences (interfaces names)
> > > as one is running inside an LXC container, listening to its
> > > single virtual interface, and the other instance is on the
> > > hypervisor/base OS listening to the bridge interface that all the
> > > containers are attached to. The container receives traffic
> > > through NAT'ing rules on the hypervisor.
> > >
> > > What I see is that certain rules aren't being triggered on the 
> > > container instance of Snort, but are being triggered on the 
> > > hypervisor. This is despite being able to see the packets that 
> > > trigger these rules appear on both machines (hypervisor and
> > > container) using tcpdump to view the respective interfaces that 
> > > Snort is configured to listen on. Specifically, the rules that
> > > I've noticed are being ignored are those that involve HTTP header 
> > > inspection, like GET /test.cgi.
> > >
> > > Like I said, I can see what look like the EXACT SAME packets on 
> > > these respective interfaces, so I've tried the following 
> > > troubleshooting without any luck.
> > >
> > >  * Switching off Snort on the hypervisor in case it was
> > > interfering.
> > >
> > >  * Creating a rule that triggers for any packet that is
> > > considered to be web traffic (i.e. EXTERNAL any -> HTTP
> > > HTTP_PORT) and this triggers for those packets without issue, so
> > > it's not a problem with those variables being misconfigured.
> > >
> > >  * Wondering whether LXC doesn't properly isolate the interfaces
> > >    somehow, so I tried configuring the container Snort to use the
> > >    bridge interface on the hypervisor, however it correctly
> > > wasn't able to use it (as it didn't exist inside the container,
> > > of course).
> > >
> > > So I'm stuck as to where to go next. The container is where I
> > > want Snort to be running, as it's my load balancer (including SSL
> > > termination) so that's where I would like to detect and block
> > > rogue traffic. The only reason that I run it on the hypervisor is
> > > to just see whether any concerning traffic is bypassing the load
> > > balancer, and whether undesirable traffic is being generated by
> > > services behind it.
> > >
> > > Thanks for your time, I really hope someone can shed some light
> > > on this frustrating situation. Very happy to answer any questions
> > > about the setup, including configuration specifics, though
> > > they're essentially vanilla installions on Debian Wheezy straight
> > > out of apt.
> > >
> > > --------------------------------------------------------------------
> > > --
> > > -------- One dashboard for servers and applications across 
> > > Physical-Virtual-Cloud Widest out-of-the-box monitoring support
> > > with 50+ applications Performance metrics, stats and reports that
> > > give you Actionable Insights Deep dive visibility with
> > > transaction tracing using APM Insight.
> > > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> > > _______________________________________________ Snort-users
> > > mailing list Snort-users () lists sourceforge net Go to this URL to
> > > change user options or unsubscribe:
> > > https://lists.sourceforge.net/lists/listinfo/snort-users
> > > Snort-users list archive:
> > > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
> > >
> > > Please visit http://blog.snort.org to stay current on all the
> > > latest Snort news!
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across
> Physical-Virtual-Cloud Widest out-of-the-box monitoring support with
> 50+ applications Performance metrics, stats and reports that give you
> Actionable Insights Deep dive visibility with transaction tracing
> using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________ Snort-users mailing
> list Snort-users () lists sourceforge net Go to this URL to change user
> options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users
> list archive:
> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users
>
> Please visit http://blog.snort.org to stay current on all the latest
> Snort news!


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://sourceforge.net/mailarchive/forum.php?forum_name=snort-users

Please visit http://blog.snort.org to stay current on all the latest Snort news!