Re: DDOS, IDS, RTBH, and Rate limiting

[Denys Fedoryshchenko <denys () visp net lb>]

On 2014-11-21 03:12, Roland Dobbins wrote:
> On 21 Nov 2014, at 6:22, Denys Fedoryshchenko wrote:
>
> > Netflow is stateful stuff,
>
> This is factually incorrect; NetFlow flows are unidirectional in
> nature, and in any event have no effect on processing of data-plane
> traffic.
Word stateful has nothing common with stateful firewall.Stateful 
protocol. "a protocol which requires keeping of the internal state on 
the server is known as a stateful protocol." And sure 
unidirectional/bidirectional is totally unrelated.

> > and just to run it on wirespeed, on hardware, you need to utilise 
> > significant part of TCAM,
>
> Again, this is factually incorrect.
http://en.wikipedia.org/wiki/NetFlow#NetFlow_support
Proof, that majority of solutions runs *flow not in software.

Cisco 65xx (yes, they are obsolete, but they run stuff wirespeed)
Aug 24 12:30:53: %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold 
exceeded, TCAM Utilization [97%]
This is best example. Also on many Cisco's if you use UBRL, then you 
cannot use NetFlow, just because they use same part of TCAM resources. 
Others, for example Juniper, are using sampling (read - missing data), 
just to not overflow resources, and has various limitations, such as 
RE-DPC communication pps limit, licensing limit.
For example MS-DPC is pretty good one, few million flows in hardware, 
7-8Gbps of traffic, and... cost $120000.
> > i am not talking that on some hardware it is just impossible to run 
> > it.
>
> This is also factually incorrect.  Some platforms/linecards do not in
> fact support NetFlow (or other varieties of flow telemetry) due to
> hardware limitations.
But still they can run fine mirroring, and fastnetmon will do it's job.
> > And last thing, from one of public papers, netflow delaying factors:
> > 1. Flow record expiration
>
> This is tunable.
In certain limits. You can't set flow-active-timeout less than 60 
seconds in Junos 14 for example.
On some platforms even if you can, you just run in the limits of 
platforms again (forwarding - management communications).
> > • Typical delay: 15-60 sec.
>
> This is an entirely subjective assessment, and does not reflect
> operational realities.  These are typically *maximum values* - and
> they are well within operationally-useful timeframes.  Also, the
> effect of NetFlow cache size and resultant FIFOing of flow records is
> not taken into account, nor is the effect on flow termination and
> flow-record export of TCP FIN or RST flags denoting TCP traffic taken
> into account.
>
> > So for a small hosting(up to 10G), i believe, FastNetMon is best 
> > solution.
>
> This is a gross over-generalization unsupported by facts.  Many years
> of operational experience with NetFlow and other forms of flow
> telemetry by large numbers of network operators of all sizes and
> varieties contract this over-generalization.
Fastnetmon and similar tools popularity says for itself.
> It is generally unwise to make sweeping statements regarding
> operational impact which are not borne out by significant operational
> experience in production networks.
"What can be asserted without evidence can be dismissed without 
evidence."

> > Faster, and no significant investments to equipment.
>
> This statement indicates a lack of understanding of opex costs,
> irrespective of capex costs.
Sweet marketing buzzwords, that is used together with some unclear 
calculations,
to sell suffering hosting providers various expensive tools, that is not 
necessary for them.
OPEX of fastnetmon is a small fee for qualified sysadmin, and often not 
required,
because already hosting operator should have him.

> > Bigger hosting providers might reuse their existing servers, segment 
> > the network, and implement inexpensive monitoring on aggregation 
> > switches without any additional cost again.
>
> This statement indicates a lack of operational experience in networks
> of even minimal scale.
>
> > Ah, and there is one more huge problem with netflow vs FastNetMon - 
> > netflow just by design cannot be adapted to run pattern matching, 
> > while it is trivial to patch FastNetMon for that, turning it to 
> > mini-IDS for free.
>
> This statement betrays a lack of understanding of NetFlow-based (and
> other flow telemetry-based) detection and classification, as well as
> the undesirability and negative operational impact of stateful
> IDS/'IPS' deployments in production networks.
>
> You should also note that FastNetMon is far from unique; there are
> multiple other open-source tools which provide the same type of
> functionality, and none of them have replaced flow telemetry, either.
Thats a power of opensource. Since FastNetMon is not only tool, worth to 
mention others,
people here will benefit from using it, for free. And i'm sure, author 
of FastNetMon will
not feel offended at all.

> Tools such as FastNetMon supplement flow telemetry, in situations in
> which such tools can be deployed.  They do not begin to replace flow
> telemetry, and they are not inherently superior to flow telemetry.
>
> Again, I'm sure FastNetMon is a useful tool in many circumstances.
> But it would be a much better idea to define FastNetMon positively in
> terms of its own inherent value, rather than attempting to define it
> based upon factually incorrect negative 'comparisons' to other
> well-established, widely-used technologies which have demonstrable
> track records within the global operational community.
I can agree only that arguing about this subject is waste of time.
FastNetMon has it's narrow specific purpose - detecting very quickly 
DDoS attacks on <10G bandwidth,
where netflow just by design cannot outperform it. But FastNetMon cannot 
be used for telemetry,
and such stuff.

---
Best regards,
Denys