Re: Stopping Spam

From: David MacQuigg (dmq)
Date: 02/07/05 

Date: Mon, 07 Feb 2005 13:34:10 -0700

On Mon, 7 Feb 2005 13:36:53 +0000 (UTC), David Webb wrote:

>In article <utid01tpnmrn83b6msecrtgs30gvnhrrjq@4ax.com>, David MacQuigg writes:
< snip >
>>To summarize the article, in case you don't have time to read it:
>>
>>1) It is possible to block emails with forged domain names, and there
>>are no significant technical barriers to doing this right now.
>>
>>2) Having valid domain names will allow anti-spam companies like
>>SpamCop to replace their current unreliable IP blacklists with much
>>smaller and reliable lists of domains, rated as to their fraction of
>>spam.
>>
>>3) A rating system based on domain names, not IP addresses, will allow
>>quick and effective filtering at the receiving end.
>>

David Webb,

Thanks for your thoughtful comments and questions.

>1) SPF just tackles the problem of forged addresses it doesn't tackle SPAM.

Correct, but since large-scale forgery is the key enabler for spam, I
still think of SPF as an anti-spam tool, rather than say, a tool for
tracing an isolated email from a smart criminal.
   
>2) As I read your proposal all mail would be channelled through "reputable"
>mail senders. Hence the number of systems able to send through those systems
>which are not under the direct control of the "reputable" mail sender will be
>gigantic. (The infrastructure of the "reputable" senders mail hubs will also
>have to be gigantic but that's another issue.)
>Any of the systems able to send mail through the "reputable" sender might of
>course be a mail sending spam zombie.

There will be some "channeling" as companies with small insecure email
servers decide among their options:
(a) Continue as is and face increasing filtering by recipients of
their email.
(b) Secure their servers and get on a whitelist.
(c) Contract with a reputable ISP to relay their outgoing email, at
least the mail that has to go to recipients that might mistake it for
spam.

I don't see this as a gigantic problem, certainly nothing comparable
to the $22 billion/year problem we are having now with spam. The
channeling will be a burden on companies that want to operate
"broadcast transmitters", but don't know how to avoid interfering with
others use of the "bandwidth". It will also be a business opportunity
for companies that want to help them pursue options (b) or (c).

>You explicitly state that: "This list would not even need to include most small
>companies, since they most often use an email server provided by their ISP.
>Even larger companies that operate their own internal email servers for reasons
>of economy or security, often use the machines of a reputable ISP for "outside"
>mail."
>This isn't true. Most companies run their own mail infrastructure - mailhubs,
>and mail stores
>So, unless you forced everyone to go through a small list of "reputable" mail
>servers you would with todays mail infrastructure have a very large list of
>valid mail sending domains.

The companies I have worked at have just one "gateway" to the outside,
and it is quite secure. The bulk of the mail is internal.

I don't have a good estimate of how many domains will need a server
with a "broadcast license" (membership on a whitelist). Maybe
500,000? I see there are now about 235,000 that have published their
SPF records, but all that takes is fifteen minutes, so I suspect most
of them are not serious yet about getting on a whitelist.

http://spftools.infinitepenguins.net/register.php
   
>How many spam mails inadvertently passed on by the "reputable" mail sender
>would it take for that "reputable" mailers domain to be blacklisted - thereby
>blocking all the users of that "reputable" mail sender from sending mail ?

The rating of domains is the major challenge each whitelist organizer
will face. Screw up, and the whitelist itself loses reputation. A
simple start would be the number of spams reported, ratioed to the
total number of outgoing emails. The handling of appeals will also be
a factor distinguishing a well-manged list. I would not immediately
put a reputable domain on a blacklist, but just lower their grade a
notch until they get their spam problem under control.

As for collateral damage to innocent users, I think the switch from IP
blacklists to domain-name whitelists will be a big help. Currently,
the IP blacklists have to block an entire IP range to block any
temporary IP that a spammer might use. When a domain name is
downgraded, it affects only the users under that name. An ISP might
have several names.

>Note. If you could get all mail to be sent via Organisation's central mailhubs
>rather than users being able to send mail directly from their clients then your
>list of domains would be identical to a list of the ip addresses of those
>central mailhubs. And of course those IP addresses would then appear on
>already existing reputable spam blacklists and anyone using those lists would
>therefore be blocking the whole domain.

An ISP could use its best name for just those flows that are known to
be safe. Alternatively, they can tell a large customer with an
outgoing spam problem to get their own domain name. IPs are easy to
acquire. Reputable names are not.

>3) How are your "reputable" mail senders supposed to distinguish between spam
>and non-spam mail ? If they use filters then those will only be partially effective.
>One person's spam is another person's important mail message.

AOL and many other reputable senders are doing it now. I'm not a
security expert, so I can't say *how* they do it, I just know they are
successful. Small companies may need to hire a security expert to
help them get outgoing spam under control.

>This is why anti-spam filter software is usually just applied to mail being
>delivered rather than to mail being sent.
>For mail being delivered you can tag the mail and then deliver it or
>quarantine it - so that the recipient can then decide what to do with it.
>You can also allow users to setup their own white/allow lists to say that
>they definitely want to receive mail from certain addresses even if it does
>look like spam.
>
>A "reputable" mail sender who automatically deletes mail which is being
>sent just because a filter it is running thinks it is spam runs a severe risk
>of being sued.

The key difference between blocking at the source and at the
destination is that the mail sender has a relationship with the
source. This would typically include an agreement to allow blocking
of spam. I can't see any risk of lawsuits. I think the discussion
would go more like:

ISP: We had to block a stream of messages coming from your <hostname>
We are getting complaints that they are spam.

Customer: If you keep filtering us, we'll find another ISP that lets
us do what we want.

ISP: You can continue to use our services, but you must get your own
domain name for outgoing mail. We will give you one of our IP
addresses with no filtering or port restrictions, and you can do what
you want with it.

Filtering at the receiving end will still be done, but it is much
easier at the source end, where the sender knows the source and can
make a judgement based on reputation and content. The receiver should
not have to filter on content, just on the reputation of the sender.

The typical mail transaction I envision has four players: source -
sender - receiver - recipient. The sender vouches for the source by
using the senders good name. The receiver filters based on the
reputation of that name and the recipient's preferences. The system
works because it is all voluntary, and it is impossible to forge
someone else's good name.

-- David MacQuigg

>David Webb
>Security team leader
>CCSS
>Middlesex University

Relevant Pages

  • Re: Alot of SMTP
    ... Exchange 2003 has the following tools for addressing spam: ... sender filtering ... recipient filtering based on user defined list ... black lists ...
    (microsoft.public.exchange2000.admin)
  • Re: Question about filtering..
    ... Exchange 2003 which comes with many more tools for filtering spam. ... 2003 with all of the filtering options (sender, recipient, blacklists, ...
    (microsoft.public.exchange2000.admin)
  • Re: Question about filtering..
    ... Exchange 2003 which comes with many more tools for filtering spam. ... 2003 with all of the filtering options (sender, recipient, blacklists, ...
    (microsoft.public.exchange2000.general)
  • Re: Question about filtering..
    ... Exchange 2003 which comes with many more tools for filtering spam. ... 2003 with all of the filtering options (sender, recipient, blacklists, ...
    (microsoft.public.exchange2000.misc)
  • MSF antispam info
    ... Spam and fraudulent e-mail messages are major issues for computer users ... Exchange Server, and Microsoft Exchange Hosted Filtering. ... and personalized spam protection while reducing false positives. ...
    (comp.mail.misc)