Re: IDS load testing

["Marcus J. Ranum" <mjr () nfr net>]

Thomas H. Ptacek        writes:
> Hello, Mr. Stolarchuk. I think it's great that the people at NFR are
> validating their product's ability to monitor a typical busy FDDI network,
> however, I'd like to point something out (you probably already realize
> this):
>
> IDS denial of service attacks that involve resource consumption do not
> necessarilly involve ping floods or smurf attacks.[...]

I asked Mike to post a few numbers, less to describe what NFR
is doing and more to show some of the other folks on the list
exactly what "a busy network" means.  :) It's been a learning
experience for us. :) I'm sure Vern would have something to say
about it, too. :)

When you're building an IDS that's intended to do some kind
of analysis on a busy FDDI you've got some problems that are
more excruciating than just looking for strings in reassembled
packets. You start running up against problems throughout the
system. At 18,000 packets per second you suddenly have a
finite number of CPU instructions you can expend before the
next packet hits!! We've also been finding fun things like
buffer starvation in some O/S' packet sucking routines, so
you silently miss large numbers of packets. :) One thing that
I've found interesting about watching Mike's measurements is
that he keeps seeing more and more packets!!! :)

Some of our early tests with NFR on a Solaris machine showed
that the engine was getting some small number of thousands of
packets per second. And the DLPI interface said there were no
drops. So we were pretty psyched. But tcpdump saw MORE packets
at the same time and also didn't see any drops. We believed
that packets were not being created. :) As we improved out
packet grabbing code and switched architectures, we started
seeing even more packets. :) It's instructive that it's taken
us this long to explore the performance envelope we want to
work in -- let alone deal with it. There are other implications
that interest us, too: Mike's seeing 15MB/sec traffic -- a
disk array capable of handling sustained writes in that area
is an expensive thing. So maybe we're on the right track that
you need to be flexible about deciding when to store the traffic
and when to just store information about the traffic.

> I just wanted to make sure that we all understand that DOS vulnerability
> is not necessarilly addressed by figures (tossed about by vendors other
> than NFR) like "capable of monitoring 20mb/s of traffic without drops!".

Denial of service is a nasty problem and, no, we weren't
trying to say it didn't apply to us just 'cuz we can suck
data real fast. :)  We hear you on that score. I think our
results show that a FDDI is a Studly Pipe and any vendor who
says casually "no problem" needs to get thier clues repaired.
I think (this is a guess, we haven't measured yet) that we'll
also find that a FDDI is, in itself, a Denial Of Service
against Windows NT. :)  We're architecting our NT stuff to look
a lot like our UNIX stuff internally but there's still a lot
of interprocess communication there and Microsoft has never
shown that they understand interprocess communication.

Back when I was in the firewall business, we used to always
have sales reps from various vendors who'd tell a customer:
"SUUUUURE, it can handle an ATM network!!!!"
What they meant was:
"You can plug it INTO an ATM network"
What Mike's been learning is that *HANDLING* an FDDI is not
for wimps. Networking studs only need apply. :)

mjr.
--
Marcus J. Ranum, CEO, Network Flight Recorder, Inc.
work - http://www.nfr.net
home - http://www.clark.net/pub/mjr