Filtering mail with procmail
Posted by Steve on Wed 14 Sep 2005 at 12:01
If you are like me you receive a lot of incoming email, too much to leave in one mailbox. Using the procmail tool you can apply filters to split your mail up into different boxes - or apply almost limitless processing to incoming mail.
procmail is a general purpose program which allows you to do almost anything to incoming email messages:
- Run commands when mail comes in.
- Auto-reply to incoming mail.
- Filter mail, for example via a spam-filter.
- Save mail into different mailboxes.
- Rewrite mail subjects.
procmail is controlled via two files:
- /etc/procmailrc
- A system-wide rule file applied to all incoming mail.
- ~/.procmailrc
- A personal rule file applied to your mail only.
To get started with procmail you need to first install it, and perhaps then enable your mail-server to use it.
To install the software run, as root:
apt-get install procmail
To allow your mailserver to invoke procmail upon your incoming mail, (to actually do the filtering, etc), will be something your mailserver documentation should mention.
For exim3, or exim4, the standard Debian package works fine. It will be setup to already detect and invoke procmail if you have a .procmailrc file present in your home directory.
For a mailserver such as qmail you could add something like the following to your users .qmail file:
| preline procmail
The magic of procmail occurs when incoming mail is filtered via the settings you add to your .procmailrc file. This rule-file can contain various things.
Most typically a .procmailrc file will include two things:
- Definitions used later.
- Actual rules.
The following is a sample file :
# # My home directory # HOME=/home/steve # # The directory to save all the mail to. # MAILDIR=$HOME/Mail # # The shell to use for command execution, and the # PATH to find external commands from. # SH=/bin/sh PATH=/bin:/usr/bin:/usr/local/bin # # Save Bob's incoming mail to "bob" # :0: *(To:|Cc:|From:).*(bob@bob.com) bob
This sample file begins by setting up some variables, and then moves into matching incoming mail. The rules in general look like this:
:0 [flags] [ : [locallockfile] ] zero or more conditions (one per line) exactly one action line
So the example:
:0: *(To:|Cc:|From:).*(bob@example.com) bob
Has no flags, one match-line, and one action. The result of this is that incoming mail which was sent To, From, or CC'd to "bob@example.com" will be saved into the folder "/home/steve/Mail/bob".
More complex operations, such as running programs to filter mail can be conducted. For example if you wish to run a SPAM-filtering program against your mail you could use something like this example:
# # CRM144 Spam Scanning # :0fw: .msgid.lock | /usr/bin/crm -u /home/steve/.crm mailfilter.crm :0: * ^X-CRM114-Status: SPAM.* crm-spam
This snippet has two sections. The first uses the "|" symbol to invoke the crm program against each incoming mail message. The message is then replaced by the output of the program, and the original unmodified message is discarded.
The intention is that when the crm program finishes any identified SPAM mail will be modified to have a "X-CRM114-Status: SPAM" header added. This line is then detected in the second section - and the relevant message is moved to the crm-spam folder for later double-checking/deletion.
The order of the rules in your .procmailrc file is significant - the rules listed first are executed first. You can take advantage of this to make sure you test for SPAM before doing anything else.
When you start running rules which might possibly drop mail, e.g. the following to simply delete matching messages, you might wish to make sure you have a backup first:
# # Unconditionally messages identified as SPAM. # :0: * ^X-CRM114-Status: SPAM.* /dev/null
I backup all incoming mail each day, before any other processing, with a rule like this:
# # Used for keeping a backup of each days mail. # BACKUP=$MAILDIR/backup TODAY=`date +%d-%m-%Y` # # Save a copy of each email received into a file of the form # '~/Mail/backup/dd-mm-yyyy'. # :0 c: $BACKUP/$TODAY
Here the "c" flag is used to make a copy of the incoming message. To make sure this works simple create the directory ~/Mail/backup like so:
mkdir ~/Mail/backup
This will give you a dated archive of all incoming mail which you can view if you suspect you've made a mistake in your .procmailrc file:
steve@skx:~$ ls ~/Mail/backup 10-09-2005 11-09-2005 12-09-2005 13-09-2005 14-09-2005
(Don't forget to delete old archives; you might find you're eating up a lot of disk space!)
For a more thorough explanation of procmail recipes, and the filtering you can conduct with them please consult the Procmail homepage.
There are a lot of good documentation, and sample rules, listed in the linked documentation.
[ Parent | Reply to this comment ]
lil work, big effect.
[ Parent | Reply to this comment ]
Although having read the "enumerating badness" spiel this evening, I feel the need to modify the MIME type filter to only allow trusted extension types, rather than refusing Windows executable types.
I think /dev/null'ing email is a very bad idea, if you don't want an email get the email servers to never accept it in the first place if at all possible.
[ Parent | Reply to this comment ]
Nice article. I can't take credit for them, but someone suggested:
:0: * ^X-Spam-Status: Yes spam :0: * ^X-Spam-Flag: YES spam :0: * ^Subject: \*\*\*\*\*SPAM\*\*\*\*\* spam :0B: * L7IthuqUttkbaI5toW/Ma9cREwXJr2bGKxWUSLB1PIIaS01RbFxQRCXoVDbDEpq4Y spam :0B: * cmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1v spam :0H: * ^Content-Type: text/html spam :0H: * ^Subject:.*drugs spam :0H: * ^Subject:.*Rolex spam :0H: * ^Subject:.*medication spam :0H: * ^Subject:.*sell spam :0H: * ^Subject:.*inch spam :0H: * ^Subject:.*\$ spam :0H: * ^Subject:.*generic spam :0H: * ^Subject:.*prescription spam :0H: * ^Subject: =?iso-8859-1 spam :0H: * ^Subject:.*buy spam :0H: * ^Subject:.*credit spam :0H: * ^Subject:.*v.agra spam :0HB: * ^Content-Type: multipart/(mixed|alternative) * ^Content-Type:.*(audio/x-|application|x-rasmol) * name=.*\.(scr|com|bat|pif|lnk|exe) spam
Then normal stuff and a local run through Spamassassin. spam is a mailbox I check for spam, before manually deleting spam every few days.
-- "It's Not Magic, It's Work" Adam
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Send Message ]
a rather radical rule would be:
#Recipe 7
#Delete all messages with exe attachments
:0
* ^content-type: application/octet-stream
/dev/null
in case you dont want em "nulled" output them to a special file...
greets oliver
[ Parent | Reply to this comment ]