Re: DDOS, IDS, RTBH, and Rate limiting

[Denys Fedoryshchenko <denys () visp net lb>]

On 2014-11-20 23:59, Roland Dobbins wrote:
> On 21 Nov 2014, at 4:36, Pavel Odintsov wrote:
>
> > I tried to use netflow many years ago but it's not accurate enough and
> > not so fast enough and produce big overhead on middle class network
> > routers.
>
> These statements are not supported by the facts.  NetFlow (and other
> varieties of flow telemetry) has been used for many years for traffic
> engineering-related analysis, capacity planning, and security
> purposes.  I've never seen the CPU utilization on even a modest
> mid-range router rise above single-digits, except once due to a bug
> (which was fixed quickly).
>
> Flow telemetry scales and provides invaluable edge-to-edge traceback
> information.  NetFlow telemetry is accurate enough to be used for all
> the purposes noted above by network operators across the world, from
> the smallest to the largest networks in the world.
>
> There are several excellent open-source NetFlow analysis tools which
> allow folks to benefit from NetFlow analysis without spending a lot of
> money. Some of these projects have been maintained and enhanced for
> many years; their authors would not do that if NetFlow analytics
> weren't sufficient to needs.
>
> Packet-based analysis is certainly useful, but does not scale and does
> not provide traceback information.
>
> > FastNetMon can handle 2-3 million of packets per second and ~20Gbps on 
> > standard i7 2600 Linux box with Intel 82599 NIC.
>
> See the comments above with regards to scale.  This is inadequate for
> a network of any size, it does not provide traceback information, and
> it does not lend itself to broad deployment across a network of any
> size.
>
> I'm sure FastNetMon is a fine tool, and it's very good of you to spend
> the time and effort to develop it and to make it available.  However,
> making demonstrably-inaccurate statements about other technologies
> which are in wide use by network operators and which have a proven
> track record in the field is probably not the best way to encourage
> folks to try FastNetMon.

Netflow is stateful stuff, and just to run it on wirespeed, on hardware, 
you need to utilise significant part of TCAM,
i am not talking that on some hardware it is just impossible to run it.
So everything about netflow are built on assumption that hosting or ISP 
can run it. And based on some observations, majority of small/middle 
hosting providers are using minimal,just BGP capable L3 switch as core, 
and cheapest but reliable L2/L3 on aggregation, and both are capable in 
best case to run sampled sFlow.
And last thing, from one of public papers, netflow delaying factors:
1. Flow record expiration
2. Exporting process
• Typical delay: 15-60 sec.
So for a small hosting(up to 10G), i believe, FastNetMon is best 
solution. Faster, and no significant investments to equipment. Bigger 
hosting providers might reuse their existing servers, segment the 
network, and implement inexpensive monitoring on aggregation switches 
without any additional cost again.
Ah, and there is one more huge problem with netflow vs FastNetMon - 
netflow just by design cannot be adapted to run pattern matching, while 
it is trivial to patch FastNetMon for that, turning it to mini-IDS for 
free.

---
Best regards,
Denys