FC: John Levine: Challenge-response systems are as harmful as spam

[Declan McCullagh <declan () well com>]

[John makes good points. There are flaws in many current C-R systems: (1) 
They rely on the From: line for authentication, (2) most current ones 
("reply to this message" or "click on this link") can be trivially bypassed 
by spammers, (3) they do not understand mailing lists. The first two are 
security problems and the third is a problem of poor integration and 
list-intelligence. Problem #1 is probably the most serious; it can probably 
be solved through micropayments, hashcash, digital signatures (web-of-trust 
or certification authority). But none of those technologies will be 
deployed in a hurry, and an alternative, keywords embedded in Subject: 
lines or the message body, is painful and awkward. --Declan]

---

Date: 11 May 2003 21:41:35 -0400
Message-ID: <Pine.BSI.4.40.0305111408240.28246-100000 () tom iecc com>
From: "John R Levine" <johnl () iecc com>
To: "Declan McCullagh" <declan () well com>
Subject: Re: FC: MailFrontier.net, poor anti-spamware, and future of 
mailing lists
In-Reply-To: <5.2.1.1.0.20030511122149.00b1a710 () mail well com>

> My reluctant conclusion is that C-R systems with flawed implementations
> have the potential to end legitimate mailing lists as we know them today.

No, it's worse than that.  The collateral damage from widely used C/R
systems, even with implementations that avoid the stupid bugs, will
destroy usable e-mail.

Challenge systems have effects a lot like spam.  In both cases, if only a
few people use them they're annoying because they unfairly offload the
perpetrator's costs on other people, but in small quantities it's not a
big hassle to deal with.  As the amount of each goes up, the hassle factor
rapidly escalates and it becomes harder and harder for everyone else to
use e-mail at all.

A relatively easy to solve problem with challenge systems is that most of
them are written by dimwits who don't understand the way that e-mail
really works.  In 1983 the 4.3BSD Berkeley Unix "vacation"  program
correctly dealt with mail from lists and other mechanical sources, yet 20
years later I still see out-of-office replies from Lotus Notes and MS
Exchange to list mail every day.  (Is there really nobody at IBM or
Microsoft who used 4.3BSD or knows the rules of thumb to recognize
non-personal but legit mail?)  Challenge systems have the same bugs, and
list managers are now routinely kicking people off lists whose broken
challenge systems spam out stupid challenges to everyone who posts to the
list, and ignoring challenges to signup confirmation messages.  These
particular problems are soluble; the few challenge systems used by
experienced mail users like Brad and Dan Bernstein avoid them.

But the real damage from challenge systems will come when spammers start
attacking them.  Challenge systems all have user whitelists so that each
correspondent only gets one challenge, then mail goes through directly. So
spammers will start trying to send spam with forged sender addresses that
are on the recipients' whitelists.  That's not so hard, sign up for a
mailing list, scrape addresses from the list traffic, then send NxN copies
of spam, to each list address from each list address.  Similarly with
addresses scraped in groups from web pages, usenet groups, and anywhere
else scrapage happens.

So what will the effect of this be?  You won't be able to trust that mail
from your friends is actually from your friends, since an increasing
fraction will be spam leaking through your challenge system.  What will
people do?  Given the basic principle of challenge systems, which is that
it's someone else's job to solve your spam problem, people will dump their
whitelists and start challenging every message.  At this point, it's
possible to automate much of the work, most challenge systems are
scriptable, so that for example I have a few lines in my mail sorting
filters that catch the per-message challenges from submissions to Dan
Bernstein's mailing lists and automatically send confirmations.  But of
course, if I can send responses from scripts, spammers can and will too,
so challenge systems will increasingly include "prove you're human"
features like showing you a picture and asking you how many kittens are in
it.  Now we'll have challenge systems duelling to the death, since
everyone will be insisting that everyone else confirm first.  There should
be ways to mitigate the damage, by using a mechanism other than e-mail for
the challenge traffic, but I don't see anyone deploying them or even
thinking about what a world where everyone challenges e-mail will be like.

So anyway, you heard it here first, challenge systems will destroy e-mail
as we know it.  Yeah, this sounds apocalyptic, but the pieces are all
falling into place, and spam problems consistently get worse faster than
anyone expects.  How many people would have predicted even a year ago that
by now there'd be more spam than real mail on the net?  Yet that's the
reality already, and the challenge juggernaut is gearing up fast.

Regards,
John Levine, johnl () iecc com, Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Sewer Commissioner
"A book is a sneeze." - E.B. White, on the writing of Charlotte's Web




-------------------------------------------------------------------------
POLITECH -- Declan McCullagh's politics and technology mailing list
You may redistribute this message freely if you include this notice.
-------------------------------------------------------------------------
To subscribe to Politech: http://www.politechbot.com/info/subscribe.html
This message is archived at http://www.politechbot.com/
Declan McCullagh's photographs are at http://www.mccullagh.org/
Like Politech? Make a donation here: http://www.politechbot.com/donate/
-------------------------------------------------------------------------