Re: 95% of User Generated Content is spam or malicious

["Tomas L. Byrnes" <tomb () byrneit net>]

Toppost and shameless plug:

Threatstop users running the default TS blocklists on their firewalls
before the anti-spam systems see, typically, 15% to 25% reduction in
average SMTP traffic, and a reduction of peak SMTP traffic to 1/4 of
what it is without ThreatSTOP. This was discovered in a specific test
run by an actual end user for a case study, and is substantiated by
other users anecdotally.

It makes sense: Since using the TS lists drops the SYN packets at the
firewall, the spambots don't see that there is a valid SMTP host there,
and stop trying after about 5 tries.

The implementation of traditional DNSBLs does require accepting the TCP
connection while the lookup is done, and then issuing the (E)SMTP 500
message, which lets the botnets simply hand you off to a host not listed
in the DNSBL.

It's not a replacement for any of the steps listed, but if you make it
the first, coarse, filter, it does reduce the amount of load in the
deeper filters, thereby reducing the needs for hardware upgrades, and
gets you a lot of your pipe, and especially a lot of your peak, back.

SMTP filtering is, of course, only one thing running IP reputation on
your firewalls gets you. The biggest security gain is the blocking of
the call home to the C&C, and the attendant alert allowing you to
remediate, from internal Trojaned hosts, IMO.

> -----Original Message-----
> From: funsec-bounces () linuxbox org [mailto:funsec-bounces () linuxbox org]
> On Behalf Of Rich Kulawiec
> Sent: Sunday, February 14, 2010 1:50 PM
> To: funsec () linuxbox org
> Subject: Re: [funsec] 95% of User Generated Content is spam or
> malicious
>
> On Wed, Feb 10, 2010 at 10:24:27PM -0500, Dave Paris wrote:
> > Where the trick (to the extent it's a trick, I suppose) lies here is
> > what it takes to knock down this volume.
>
> I use my firewalls/routers.  Beginning with the DROP list, followed up
> by a large number of country, so-called ISP/webhost (spammer front,
> e.g., Eonix), so-called ESP (spammers, e.g., iContact, Uptilt) blocks.
> And I use passive OS recognition to treat anything that's running
> Windows differently -- because the the odds are in the 10e4-10e6
> to 1 range that it's not a real mail server, depending on how I
> construct the metric.
>
> Then I enforce DNS/rDNS existence and consistency checks on the
> connecting
> host and the HELO parameter.  Then I use a large set of rDNS patterns
> that's been very carefully developed to match non-mail-sending
> hosts (e.g. end-user systems) and refuse everything from them
outright:
> real mail systems have real names, not generic ones.
>
> Then I use a local blacklist of domains, sender LHS, senders, hosts,
> networks, etc.  Then several DNSBLs.  And a few other things.
>
> This is an extremely effective, efficient and very accurate setup.
> It's effective because it doesn't waste time trying to figure out if
> the same abusers who sent spam yesterday are sending some more today:
> of course they are.  It's efficient because it rejects/accepts
outright
> -- and when it rejects, it does so before seeing the message-body.
> It's accurate (a) because it's based solely on deterministic criteria
> (b) because I've been doing this for a very long time (sadly) and
> have learned a few things by now and (c) because it's correlated
> against
> my own mail logs, a necessary but seldom-performed step that helps me
> understand what the spam and non-spam components of my mail stream
are.
> I've gone into considerably more detail about this on mailop, if you
> want the extended writeup, but the bottom line is that this kind of
> approach beats the pants off more complex/trendier ones in terms
> of performance, simplicity, resistance to attack, FP rate, FN rate,
> maintainability and predictability.  But it does require pretty
> good knowledge of what *your* spam/not-spam mail mix looks like:
> you've got to understand it well before deploying something like this.
>
> ---Rsk
> _______________________________________________
> Fun and Misc security discussion for OT posts.
> https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
> Note: funsec is a public and open mailing list.

_______________________________________________
Fun and Misc security discussion for OT posts.
https://linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.