Click the comments link on any story to see comments or add your own.
Subscribe to this blog
23 Dec 2013
Everyone who's been in the e-mail biz long enough knows the term FUSSP, Final Ultimate Solution to the Spam Problem, as described in a checklist from Vern Schryver and a form response that's been floating around the net for a decade.
FUSSPs fall into two general categories, bad ideas that won't go away, and reasonable ideas that are oversold. The classic bad idea is e-postage, charging for e-mail. I wrote a white paper about it a decade ago, and nothing of importance has changed.
Reasonable ideas that are oversold are much more frustrating, and there are a lot of them in the e-mail authentication field. One of the bad ideas is that if we just knew who was sending every mail message, we'd have no spam, which is silly for many reasons.
It's not at all silly in weaker versions. There are plenty of partial authentication schemes that let us identify interesting parts of our mail stream so we can whitelist mail from some senders we know. Attempts to give an identity to each user, PGP and S/MIME, have never been widely adopted because it's too hard to distribute the keys into each user's mail program. Less ambitious schemes where the granularity is the domain name are quite workable for parts of the mail stream, but suffer when they're oversold.
A sure fire way to tell when an authentication scheme is turning into a FUSSP is when naive enthusiasts demand that other parts of the mail system that work perfectly well have to change what they do to deal with limitations of the authentication scheme. Invariably, the mail system keeps doing what it's always done, and people learn to live with the limitations of the authentication scheme, but teaching the enthusiasts to curb their enthusiasm is very frustrating.
About a decade ago, the new authentication scheme was SPF. For some kinds of mail, particularly bulk mail all sent from a known set of servers, it works quite well. For others, such as mail systems with human users who send mail in a variety of hard to predict ways, it works much less well. SPF has ways to deal with this limitation, by allowing various intermediate results between pass and fail, but by golly the enthusiasts wanted to fix the e-mail world so that all legitimate mail would pass. (Presumably they would reject all mail that didn't pass, and solve the spam problem.) To try to force the issue, some of them had their mail systems reject any mail that didn't pass, thereby losing lots of perfectly ordinary mail that didn't happen to be sent in a way that SPF can describe. Add-ons to SPF with names like SES were supposed to let the rest of us "upgrade" our mail systems so that forwarded mail, a particular sore point, could be SPF compliant, but in practice what happened was that people adjusted their SPF to avoid needless damage and the mail system works the same as always.
The next round was with DKIM. While SPF tries to validate the path a message takes, DKIM puts a cryptographic signature in the message. That doesn't depend on the path the message takes, so forwarding isn't usually a problem. For DKIM, the FUSSP issue arrives with mailing lists. Most mailing lists make small changes to the messages for the convenience of the list subscribers, such as adding a list tag to the subject line, or a message footer that says what the list is and how to subscribe or unsubscribe. If the incoming message has a DKIM signature, these changes invalidate it. The mail system running the list software can add its own DKIM signature, recipients use that signature to recognize list mail, and everything works.
Some DKIM enthusiasts insist that mailing lists need to preserve DKIM signatures from incoming mail by turning off the convenience features that break signatures, even though that would solve no problem whatsoever. (I can't wait to see the comments on that one.) List operators, of course, have ignored them, since the subject tags and message footers are useful, and lists work just fine as is.
This year's authentication hack is DMARC. It sits on top of DKIM and SPF and allows senders to describe their sending policies, and offer advice to quarantine or reject mail that doesn't match the policy. For domains like Paypal that are heavily forged and send all their mail from a small set of known hosts, DMARC is very useful to tell people how to detect forged mail. For domains like mine, who are forged only by the occasional random spammer who picks fake return addresses out of the spam lists, and whose mail goes through forwarders, mailing lists, newspaper send-to-a-friend, and so forth, DMARC has some interesting ways to gather statistics, but the policy parts are not useful.
Predictably, there are DMARC enthusiasts who want every message ever sent by anyone to have a "reject" policy, which will solve the spam problem by rejecting all spam. (Perhaps they are too young to remember the past two iterations.) This again brings us back to mailing lists. DMARC policy assertions can't describe list mail for the same reasons that list mail fails DKIM, and it doesn't matter for the same reason, receivers can recognize mail from lists their users subscribe to. But DMARC adds an extra level of excitement: overzealous use of DMARC by list contributors can make receivers reject valid list mail, which list software interprets as bad recipient addresses, and removes entirely valid subscribers from the list.
You will probably not be surprised to hear that the enthusiasts blame the problem on the mailing lists, which should turn off features that break DKIM signatures, or need to be rewritten with complicated heuristics to recognize and ignore DMARC bounces, or something, rather than the actual problem, which is that they're publishing the wrong policy. List software will of course not change what it does; at most it will recognize and reject incoming mail from subscribers that have strict DMARC policies. There are already patches to GNU mailman to do this, and I've found I can adjust my majordomo2 lists do to so with a one line configuration change.
I suppose it's nice to know that we're still getting new enthusiasts in the anti-spam biz, but it sure would be nice if they'd read the history before making the same mistakes over again.
comments... (Jump to the end to add your own comment)
industry observer, and serial napper
Looseness brings no assuredness
Add your comment...
Note: all comments require an email address to send a confirmation to verify that it was posted by a person and not a spambot. The comment won't be visible until you click the link in the confirmation. Unless you check the box below, which almost nobody does, your email won't be displayed, and I won't use it for other purposes.
My other sites
© 2005-2018 John R. Levine.
CAN SPAM address harvesting notice: the operator of this website will not give, sell, or otherwise transfer addresses maintained by this website to any other party for the purposes of initiating, or enabling others to initiate, electronic mail messages.