Re: newbie quetsions (on how much Snort sucks)

[Martin Roesch <roesch () sourcefire com>]

On Jan 7, 2005, at 1:03 PM, Dave Aitel wrote:

> Martin Roesch wrote:
>
> > Dave,
> >
> > I'm interested to know if you think Snort's stream reassmbler can't  
> > handle your 1-byte segment test *by design* or as the result of a  
> > *bug*?
>
> I had to assume it was by design. Is it a bug?

I don't have any data to work with so it's hard to say.  I'd bet it's a 
side effect of the way that we do "flushing" in the stream reassembler, 
a simple fix would be to either make the flushing point larger (so we 
can buffer more one byte segments to get a complete DCERPC record) or 
by changing the behavior of the flushing system.  The new stream 
reassembler system I'm developing (stream5) allows you to set default 
global flushing policies on a per port basis, user defined flushing 
policies per port & IP, custom policies per port/IP, and programmatic 
policies from both the preprocessors and the rules language.  These 
policies also let you determine if you're going to do sliding 
window-based presentation to the analysis engine, snapshot-based or 
query/response-based.  I'm not 100% sure, but I think that should cover 
the bases pretty well.

> Well, a Snort box is useful for network monitoring and to look for 
> worms, even if you aren't using it to look for hackers. Most CIOs 
> really don't care about hackers, and do care a lot about worms, so I 
> think in some senses, the marketplace doesn't differentiate.

To a large degree I would concur about the CIO comment and I think a 
properly tuned, well deployed Snort sensor is great at picking up 
"hackers".  You get out of IDS what you put into it.

> Likewise, a lot of the new technologies Sourcefire is working on are 
> really cool.

Thanks.

> > That being the case, if you were in the "open  source spirit" I would 
> > probably expect to see a bug report someplace  like snort-users or 
> > snort-devel or even in my inbox rather than blanket  statements like 
> > "Snort's stream reassembler is horrible because it  failed my test 
> > case" in forums like this one.
>
> Aside from the presentation from October on Immunity's web site, the 
> public announcement on this list, and the many announcements to the 
> CANVAS mailing list dating back at least a year, I had to assume 
> Sourcefire had done the basic QA.

Sorry, I haven't been very diligent about tracking your work in this 
area until this popped up.  You made an announcement on CRI back in 
October on the mailing list that made no mention of Snort.  We knew 
about the DCERPC frag issue before that, but that's a more complex 
issue to be solved with a preprocessor.

> And of course, that's all assuming the attacker doesn't get a copy of 
> your IDS engine and do real analysis to evade you, which I haven't 
> done, and don't plan to do.

One of the concepts of this idea of being "target-based" I keep going 
on about is that we can know enough about the execution environment to 
model things properly even if the bad guy has our engine.  The new 
Snort IP defragger (frag3) is built around this idea and products like 
RNA exist to gather and distribute the necessary context required to 
run these engines.  It's not a 100% solution but it's a start.

> > You could even take an  extra step and actually help out (really 
> > getting into the open source  spirit now!) by making a simple pcap of 
> > the failed session so that we  could do the debugging for you and let 
> > you know what's going on if you  didn't want to take an hour and 
> > figure it out yourself.
>
> Everyone I've talked to has assumed it's a "feature" of the engine. I 
> assumed you guys knew about it. I personally find the snort signature 
> language nearly impossible to read, so I didn't push deeply into it.

Now the Snort rule language is something that you could rightly point 
to and say that it "sucks" if only because it has become unreadable.  
We're working on that, the Snort rules language today is a victim of 
its own success.

Anyway, it's not a feature of the engine, it's a problem and should be 
treated as such.  A pcap of the traffic that you're generating would be 
helpful so I don't have to setup an execution environment that you 
already have up and running.

> > Now, just off the top of my head I suspect I know what the problem 
> > is,  but really, couldn't you do anything more than just show up here 
> > and  talk about how badly we suck?
> It's my job to say when defenses are weak. That's all my company does. 
> It's what people need to know when they're deploying defenses. The CRI 
> is a reproducable test everyone can use for free, both customers and 
> vendors. Neither I nor my company is taking money to bash or promote 
> any particular IDS or IDS technology.

But when open source technology is involved, part of the compact of 
"community" that is a key component of making open source work is that 
you actually help to solve the problem.  All we need is a pcap and a 
filled in BUGS form (which comes with Snort) and we do the rest for you 
and everyone else in the community for free.  I'm a busy guy too, meet 
me half way.

> > As for the fragmented DCERPC records, you're right, you got us there. 
> >   Interestingly enough, we have made allowances for just this sort of 
> >  thing in Snort by building several APIs that allow you to extend  
> > Snort's functionality in case something like this comes along that we 
> >  didn't think of when we first developed Snort.  In this particular  
> > case, I'd say we need to normalize the DCERPC calls which would  
> > indicate to me that a Snort DCERPC normalization preprocessor would 
> > be  the appropriate route to solving this problem.
>
> Immunity doesn't sell a Snort-based product.

So what?  Snort's an open source project that Sourcefire just happens 
to run and do an inordinate amount of work on, why can't you contribute 
as part of the user community?  If this is Immunity publicizing 
problems with a product, why didn't we get a formal notification as a 
courtesy so that we could come up with either a patch or a remediation 
strategy?  Do you do that?  I looked around your website and didn't see 
if you have a vulnerability disclosure policy that's modeled on 
something like RFPolicy or something more along the Pearl Harbor lines. 
 Is there such a policy?

> If we did, we no doubt would have to go into Snort and build all of 
> this. Realisitically it would make Snort a little bit faster, since it 
> would be running one rule for each RPC bug, instead of six.

Our rules engine doesn't work that way anymore.  Having 6 rules defined 
to detect something does not necessarily assume that the rules are 
being run for every potentially applicable packet.  I digress.

> > P.S. I've been working on a new stream reassembler since November  
> > that'll be introduced to Snort RSN.  If you look at the new IP  
> > defragmenter that I implemented which was checked into Snort CVS back 
> >  in November, you can probably get an idea where I'm headed with the 
> > new  stream reassembler.
>
> Is this even an IP-level flaw? I assumed it was somewhere else. It's 
> funny that the major benefit of the Open Source development model 
> claims is "lots of QA".

No, you missed my point.  It's not an IP-level flaw, although you'd 
probably be amazed how weak most IP defraggers are in just about every 
IDS/IPS out there today.  The 1-byte segment problem you saw is in 
stream4 (our current TCP stream reassembler).  The thing I was trying 
to illustrate is that:

1) Snort's got a brand new IP defragmenter
2) If you look at its feature set you can see it has some pretty 
cutting edge features
3) If you look at its implementation you can see it's pretty much a 
ground up reimplementation of the capability vs. frag2
4) If you configure it correctly and test it, you'll see that it works 
very well

The point that I was trying to make was that I'm working on a new 
stream reassembler that will follow along those same philosophical 
lines.  I believe that open source does tend to provide superior QA 
over time in many areas, but the number of people out there who are 
capable of doing the types of things you've been working on are a 
relatively small group of people.  We rely on people with skills like 
you to work with the community to help improve things too.

Anyway, if you'd like to do me a favor and stop saying that Snort is 
horrible because it doesn't do XYZ I'd appreciate it.  The same could 
be said about any network traffic analysis engine that has a design or 
implementation compromise that renders it incapable of detecting 
certain threats under certain conditions, and that would be all of 
them.  We've done sufficient competitive analysis and over the past 
three years to verify that that's the case in every product out there 
(even the ones you laud) so I don't see why Snort should have to be 
singled out.

     -Marty

--
Martin Roesch - Founder/CTO, Sourcefire Inc. - +1-410-290-1616
Sourcefire - Discover.  Determine.  Defend.
roesch () sourcefire com - http://www.sourcefire.com
Snort: Open Source Network IDS - http://www.snort.org


--------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------