Topics

• W3C as a Case Study
• Background on the Problem
• Counter Measures
• Dialog on Tactics

W3C - Open Standards Organization

The World Wide Web Consortium (W3C) is an Open, International Standards Organization for the Web comprised of Technical Staff, numerous Member Organizations, and scores of individual Invited Experts.

Email correspondence with such a diverse and distributed participation group is a vital part of the Consortium's operations. Besides regular contributors we accept public comments on our Technical Specifications during last call periods in particular and in general welcome feedback from the Web Community.

Mailing Archives

We run many hundreds of Mailing Lists with thousands of recipients and portions publicly archived. As such we are particularly attractive to spammers.

http://lists.w3.org

• Served with Jigsaw
• Mail Archive Search Engine
• RSS Feeds
• 700,000 views/day of the 1.8M mails in our archives
• W3C has page rank of 10

Quick Background on Spam and Virus

There are numerous writing on this subject, a good high level article on the causes, history, attempts to legislate and counter measures is available at The New Yorker.

• >90% of email correspondence estimated to be SPAM
• low cost marketing
• Sizable Botnets on Broadband and Offshore havens
  Some ISPs are starting to block outbound port 25 (SMTP) from their subnets and some organizations are outright firewalling subnets for entire countries.
• Spam is only one element of their business
• Ever-evolving spammer techniques and counter-techniques
• Email is meant to be a reliable means of communication, spam volume and counter-spam techniques have adversely affected the reliability.

Email Address Harvesting

It is often only a matter of time before any email alias is harvested by spammers. No need to make it easy for them though, increase their cost.

• Web Crawlers and Obfuscation
  
  • ted _at_ w3 _dot_ org
  • ted@w3.org <a href="mailto:ted&#x40;&#0119;&#0051;&#0046;&#0111;&#0114;&#0103;>
  • Mailback Form
• Dictionary Attacks
  Spambots iterate through a dictionary of common names querying your mail server if such an alias exists. This can be countered with real-time log analyzers like fail2ban probably before any aliases are harvested but beware of log injection risk.
• Viruses and other compromises sometimes harvest users' Address Books
• Honeypots
  Use the knowledge aliases get out to your advantage, make some honeypots. Participants in Project Honey Pot can get at the collected data in order to block crawler IPs to block from their sites.

"SMTP Consortium" - Hugo Haas

Disclaimer: W3C is not responsible for the mess that is Email.

Partial View of our Mail Processing:

• Sending server speaking protocol properly?
• Sending domain a valid domain?
• Forgery prevention data in DNS?
• Contains certain binary attachments?
• Matches known viruses signatures?
• Are they on our local blacklist?
• From a collaborator who specified their own forgery prevention data?
• Passes spam-detecting heuristic tests?
• Delivery to Staff or Mail Lists
• Archive Approval and First Time Posters systems
• Some spam does get through; for those we have annospam (append ,spam to view on specific message)

Arrange your Defenses Wisely

• Spam is an epidemic and the other side has computing resources (botnets mentioned earlier) that dwarf what you can throw at it so be careful what your first lines of defense cost in cpu.
• Some days see significant spikes in volume and you do not want it to be disruptive to your organization.
• Find or do your own benchmarks of the various tactics so you can place the less expensive ones earlier on.
• Reject mail don't send it quietly to the bit bucket (/dev/null). If your organization blackholes mail seen as spam prospective clients whose mail is improperly categorized will assume they are being ignored.
• Do what makes sense for your organization and the way it conducts business.

Reject Stats for a Week

Choose your Public Mail Server Carefully

The Mail Exchange (MX) for your organization available to the public can and perhaps should be completely different from what it's users connect to.

Enter Exim. Unlike mail servers (Mail Transfer Agent - MTA) of the past it is a bit more wary.

• Sending domain valid?
• Sender exist as recipient at MTA found from DNS MX?
• This server flood us recently?
• Reasonable to configure and add additional counter-spam techniques
  Nowhere near as cryptic as sendmail.cf
  

  	  Mprog, P=/bin/sh, F=lsoDq9, T=DNS/RFC822/X-Unix, A=sh -c $u
  	  M*file*, P=[FILE], F=lsDFMPEuq9, T=DNS/RFC822/X-Unix, A=FILE $u
  	  M*include*, P=/dev/null, F=su, A=INCLUDE $u
  	  

• It also manages a busy mail queue with temp fails more reasonably and thereby causes less load

Forgeries

• If possible have rigid policies for how your organization's users can send email from the organization domain, reject everything else pretending to be from your domain.
• Make these policies public so other organization's mail servers can reject forgeries from your domain
  • Sender Policy Framework (SPF)
  • DomainKeys Identified Mail (DKIM)
• One of the many spammer tactics is to send mail from forged addresses within an organization to other recipients within the organization.
• The more adapted spammers also utilize available data (links and other data on the web, harvested address books, etc) so as to forge spams from organizations that tend to do business with one another.

Besides reducing spam within your in-box you want to protect your and your organization's image.

Viruses

Let's face it, some mail clients and operating systems are more susceptible to viruses than others. Designing intentional hooks for email to interact with the rest of the system comes at extreme risks. Coding limitations to these interactions will be an uphill battle. There should be zero trust for something anybody can send you. People are often duped, aka Social Engineered, into opening attachments and willingly forward cutesy or humorous attachments to friends and family.

	  YOU HAVE RECEIVED THE UNIX VIRUS!

	  This virus works on the honor system. Please randomly delete
	  some of your files and forward this to everyone you know.
	

-- UNIX Joke Virus

There are many free (as in source and free as in beer) and commercial anti-virus systems operating at the server or personal computer level. Some of the prominent ones are contrasted in LinuxWorld Fight Club.

Use one at your mail server and have it update it's signature database regularly. Long before LinuxWorld's rating, we've been rather pleased with ClamAV .

Using data to fight spam
White, Black and Gray-listing

In addition to the data available on the net take advantage of data derivable from your own mail, this is the information age after all.

• whitelists
  Automating collection for an organization is worthwhile, you do not want to inconvenience or ignore your customers.
• blacklists
  There are many automated blacklists out there, unfortunately they are not completely reliable and prone to having flawed data. As such automated blacklists should only be used as part of a scoring system and not sole grounds for dismissing mail.
• greylists
  Unknown senders are asked to try back later, taking note of a few data points (sender, IP address, recipient) and spammer mail servers often only partially implement SMTP do not bother retrying anything that initially or permanently fails. If you are willing to forgo email's near instantaneous delivery for new correspondents this is very effective.

In addition to these lists approaches there are honeypots already mentioned and Bayes filtering. Bayes is statistical analysis based on categorization of contents (message and headers) of bodies of known spam and ham (legitimate email) so that a spam probability can be assigned to incoming mail. Bayes can be part of a heuristics system on your mail server that is regularly fed ham from your outgoing mail and spam from your honeypots.

Probable Spam

Not all mail easily falls into ham or spam. If there is any doubt it is preferred to categorize the uncertain and let the end user decide for themselves.

• DSPAM
  
• Spamassassin
  

Ever-evolving spammer techniques and counter-techniques

It is a vicious circle of spammers countering the counter-techniques. Looking to maximize their distribution some regularly run their spams through gauntlet of counter spam systems to see how they fair.

Bayes filtering was so effective (mid to high 90s percentage) that spammers send many millions of innocent messages without any product uri for the sole purpose of polluting bayes databases. DSPAM's algorithm is less susceptible to bayes pollution.

PDF and image attachments being the body of the email was a new technique for awhile this last Summer.

Unless they want to be inundated with Spam organizations pretty much have to have postmasters and system administrators who stay somewhat abreast of emerging trends and/or involve consulting or commercial resources to assist with defenses.

http://www.spamconference.org/ it's free and there are webcasts for those who cannot be present physically.

Do not go it alone, long gone are the days administrators and developers can cost effectively maintain their own concocted filters.

Consider Spam and Virus systems that auto-update and take advantage of Open Source Developer and User Community.