Spam filtering with qpsmtpd

Posted by Steve on Mon 30 Jul 2007 at 12:21

There are many methods used to fight spam which are tied to particular mailserver implementations. This means that unless you're using that specific software you cannot take advantage of them. A simple means to adding additional anti-spam checks to your mailserver is to place it behind an SMTP-proxy. One common proxy is the extremely flexible qpsmtpd server.

The idea of using a SMTP proxy in front of your mailserver certainly isn't a new one. It allows you to do a lot of filtering, (and testing of new techniques), without having to modify your core mailserver which is almost certainly written in a low-level language like C. It also allows you to do site-wide processing without requiring your users to implement their own filters with procmail, etc.

qpsmtpd is a flexible SMTP proxy which is written in Perl, it is designed to connect between the public internet and your mailserver, processing incoming mail before passing on valid mail mail to your real server.

Before we get started installing the proxy it is worth thinking about how it will work, and how you will test it.

The intention is that the proxy will listen upon port 25 of your public IP address, such that incoming mail may be delivered to it. If mail is filtered and found to fail any of the checks you define it will be bounced, but otherwise it will be passed on to your real mailserver. (exim, qmail, postfix, sendmail, etc.)

For this to work you will need to reconfigure your current mailserver so that it no longer listens upon port 25 of your public IP. I'd suggest you configure it to listen upon 127.0.0.1:25.

Then you can configure the qpsmtpd server to listen upon your public address and pass valid mail on to 127.0.0.1 - where it will be processed for delivery.

During testing you can avoid changing your working SMTP server. Instead configure the proxy to listen upon 127.0.0.1:2500, for example, and test against that until you're happy that things are working as expected.

Obviously before you can test anything you must install the package, and installing it upon Debian GNU/Linux machines is very simple as:

root@etch-builder:~# apt-get install qpsmtpd
Reading package lists... Done
Building dependency tree... Done
The following extra packages will be installed:
  libdigest-hmac-perl libdigest-sha1-perl libmail-spf-query-perl
  libmailtools-perl libnet-cidr-lite-perl libnet-dns-perl libnet-ip-perl
  libsys-hostname-long-perl libtimedate-perl liburi-perl
Suggested packages:
  libwww-perl spamassassin clamav-daemon tinycdb
The following NEW packages will be installed:
  libdigest-hmac-perl libdigest-sha1-perl libmail-spf-query-perl
  libmailtools-perl libnet-cidr-lite-perl libnet-dns-perl libnet-ip-perl
  libsys-hostname-long-perl libtimedate-perl liburi-perl qpsmtpd
0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
Need to get 769kB of archives.
After unpacking 3109kB of additional disk space will be used.
Do you want to continue [Y/n]? 

Once installed you will find a number of configuration files in the new directory /etc/qpsmtpd. If you weren't prompted by debconf to perform some minimal setup at package-installation time then you can configure it now by running:

root@etch-builder:~# dpkg-reconfigure qpsmtpd

This will ask you a series of questions about your mailserver - and assuming that you merely wish to forward accepted mails to 127.0.0.1:25 the defaults should be reasonable.

Once this is complete you can test your server by telneting to port you've configured it to listen upon (2500 in testing, 25 when live) - the greeting should look something like this:

skx@etch-builder:~$ telnet 192.168.1.100 25
Trying 192.168.1.100...
Connected to 192.168.1.100.
Escape character is '^]'.
220 etch-builder.my.flat ESMTP qpsmtpd 0.32 ready; send us your mail, but not your spam.

This shows that the proxy is listening, and if you direct mail through it then this should be delivered.

Interesting things start happening when you examine the configuration and enable plugins which will test the mail as it is delivered. The configuration file used to enable plugins is /etc/qpsmtpd/plugins, and it is pretty well commented.

By default the file installed will be pretty minimally configured - with most of the checks disabled. To start adding new tests to your server you will need to adjust the plugins file. As an example you can drop clients which start sending traffic to your server without waiting for the prompt to be sent, this can be done by enabling the "earlytalker" plugin:

check_earlytalker action deny

Another simple thing you can do is block clients which attempt to impersonate your own servers with bogus HELO lines. (We've previously seen how to implement HELO checking for Exim4.)

The HELO checking is enabled by default, but with an empty list. All you need to do is add bad entries to the file /etc/qpsmtpd/badhelo and restart the proxy.

Each of the settings we've introduced so far has been implemented as a small plugin located in the /usr/share/qpsmtpd/plugins directory and a lot of the power of using qpsmtpd comes from the ability to implement your own plugins.

With this simple framework it becomes very simple to write code to perform a lot of checks, for example deny all mail from IPs which have attempted to send mail to the spamtrap address of spam@example.org. If there is any interest in writing custom plugins I'd be happy to work through an example in a future article.

 

 


Posted by Anonymous (62.180.xx.xx) on Mon 30 Jul 2007 at 15:17
Does this work with smtp auth ?

[ Parent | Reply to this comment ]

Posted by Steve (89.16.xx.xx) on Mon 30 Jul 2007 at 15:19
[ View Steve's Scratchpad | View Weblogs ]

[ Parent | Reply to this comment ]

Posted by Anonymous (62.180.xx.xx) on Mon 30 Jul 2007 at 15:25
Thanks for you quick response,
but we would need some kind of smtp auth path through, because were are running an imail 2006 server :( ( i know crappy software.)

[ Parent | Reply to this comment ]

Posted by kroshka (66.252.xx.xx) on Tue 31 Jul 2007 at 00:00
[ View Weblogs ]
I would suggest using ASSP: http://www.asspsmtp.org/wiki/Welcome
It is a very effective spamfilter, once it has learned and adapted to your specific spam.

We use it here on several servers to filter incoming email, email which passes filtering will be redirected to a standalone email server running exim4 (listening on the usual ports). The assp servers are in the mx records, the email server with exim4 is not. I tell my users (this is necessary for those outside the local network) that in order to use smtp authentication they should use the email server directly for outgoing email, I have configured exim4 to allow smtp authentication.

[ Parent | Reply to this comment ]

Posted by Anonymous (83.145.xx.xx) on Mon 17 Sep 2007 at 12:02
ASSP can grow VERY slow with large number of transferred messages, however qpsmtpd hasn't show such behaviour so far and there are roughly 10k users in our mailservers.

[ Parent | Reply to this comment ]

Posted by kroshka (66.252.xx.xx) on Mon 17 Sep 2007 at 19:17
[ View Weblogs ]
"ASSP can grow VERY slow with large number of transferred messages"

Besides throwing more cpu power at it, maybe there are dns (blacklist) time outs (can be avoided using the right, often default config). Also it works well to have multiple assp servers setup in a dns record so they'll be accessed in a round robin manner. Then you'll divide the load to multiple servers.

[ Parent | Reply to this comment ]

Posted by oDn (84.101.xx.xx) on Wed 1 Aug 2007 at 13:32
Hi Steve,

thanks for this nice little article.
I was wondering though, how does this proxy scale when under heavy load?
Would it seriously bog down performance since it's written in plain old Perl?

Cheers.

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Wed 1 Aug 2007 at 13:36
[ View Steve's Scratchpad | View Weblogs ]

It scales pretty well. Especially if you add "cheap" filtering ahead of things like amavis/spamassassin which are notorious CPU hogs.

Steve

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 674 votes ~ 10 comments )