Ask SIPB - March 14, 2003

Like all e-mail accounts, Athena accounts tend to receive lots of spam. Further compounding this problem are the numerous mailing lists available, each receiving its own junk. The inconvenient and unwanted spam can be controlled, though, so this week, we explore MIT Spam Screening.

Question: What is MIT Spam Screening?

Answer: MIT Spam Screening is a piece of software recently installed to allow users to control the junk mail they receive. Based on the open source product SpamAssassin (http://www.spamassassin.org/), MIT Spam Screening uses a set of rules to give each incoming e-mail message a spam score. Messages with score greater than a certain threshold get marked as spam, allowing users to deal with them appropriately.

MIT Spam Screening tags your email so that you can filter and delete messages that might be spam. Importantly, it's "you can": you don't have to if you don't want, and if you take no action, you will continue to receive all your email as you always have.

We mention this caveat up front because of how MIT Spam Screening works. Typically, the software works by looking features often present in spam messages, for example, the phrase "Click here to unsubscribe," whether it is written in a foreign language like Korean, or uses well-known "open mail relays" (frequently in foreign countries). If enough spam-like features are found, the message is tagged.

Unfortunately, deploying Spam Screening for a diverse place such as MIT is very difficult. For example, many students receive lots of legitimate email in foreign languages from foreign countries. The moral of the story is: if you do take advantage of the filtering capabilities described below, please be sure to do a cursory check your suspected spam before deleting it.

Question: How do I start screening spam?

Answer: If you are using an IMAP mail client, such as Evolution, Mozilla, Outlook, or Athena Pine, you can have all messages marked as spam filtered into a separate folder automatically. Simply create a new folder in your INBOX named "Spamscreen" (without the quotes). Warning: If you create such a folder, you will not be able to use POP mail clients, such as Eudora, SIPB Pine, and nmh, to view email tagged as spam by the po servers.

On the other hand, if you are using a POP mail client, such as Eudora or SIPB Pine, you can set up filters to move your mail to a separate folder. Using your mail program's filters, move all messages with the header "X-Spam-Flag: YES" into a folder you specifically designate for spam.

For more detailed instructions on setting this up, please see http://web.mit.edu/is/help/nospam/. Note that the directions at the webpage do not cover SIPB (POP) pine (although they do cover Athena (IMAP) pine, which is the more commonly used version). (At the time of the writing of this column, SIPB pine is version 4.30L and Athena pine (for Athena 9.1) is version 4.33L as reported in the upper left hand corner of pine's screens.) If you do use SIPB pine, type "SRF" at the main menu to Setup a Rule for a Filter. You will want to filter for the AllText pattern "X-Spam-Flag: YES" as mentioned above.

Question: How can I screen more aggresively or tweak the settings?

Answer: The web page at http://web.mit.edu/is/help/nospam/ has links to the site where you can configure your MIT Spam Screening settings. MIT Certificates are required to access the page.

Once there, you can choose your scoring threshold for incoming mail. The default level of 7.5 will catch most spam, though you can adjust the level as necessary, especially if you determine that many wanted e-mails get marked as spam, or much spam still lands in your INBOX.

You can also tell the MIT Spam Screening program to allow all e-mail from certain users, or deny all e-mail from users. In these cases, regardless of the spam score of the messages they send, they will be marked appropriately.

Finally, on this same page, you can enable or disable automatic purging of the Spamscreen folder. If you are using an IMAP mail client (and the associated spam screening solution), you can have messages in the Spamscreen folder deleted automatically after two weeks. If you enable this option, be especially sure to check the Spamscreen folder to ensure that no messages have mistakenly been filtered there. Once they get deleted, you will NOT be able to retrieve them.

Question: How well does MIT Spam Screening work?

Answer: The MIT Spam Screening software usually works well. Reports vary, but it should catch 90% of your spam. Its default setting is conservative, that is, you will still find spam in your inbox somewhat regularly. It should only very rarely mark legitimate messages as spam (but it sometimes will, so be careful!) Be sure you check your spam folder from time to time to ensure you are not missing any important mail.

Question: I use MH for mail, but the I/S page does not explain how to to filter my spam with MH. How can I do so?

Answer: Add the following lines into the .cshrc.mine file in your home directory:
alias inc 'athrun sipb spam-inc'
The next time you login, you will be able to use inc, and the messages marked as spam will be automatically be filtered for you into a spam folder. spam-inc is a shell script in the sipb locker. You can make a personal copy of the script, if you want to customize its behavior.


To ask us a question, send email to sipb@mit.edu. We'll try to answer you quickly, and we can address your question in our next column. You can also stop by our office in W20-557 or call us at x3-7788 if you need help. Copies of each column and pointers to additional information are posted on our website: http://www.mit.edu/~asksipb/