A professional mail server with qmail and vpopmail
Posted by Derevko on Wed 12 Jul 2006 at 10:47
Qmail is a good solution for an email server, but I think the current official qmail-src package is outdated and not good for using on a modern mail server. That is why I created an unofficial qmail-src package with some suitable patches.
The package contains the following patches:
- SMTP-AUTH for Debian Sarge 3.1 (Fixed)
- qmail-dnsbl patch (added logging)
- qmail-queue-custom-error.patch (for simscan)
- qmail-smtp-log-patch
- chkuser 2.0
The qmail-dnsbl patch lets the client authenticate (using any method implemented), and then decides to perform the DNSBL check looking at the authentication status before the DATA SMTP command is performed.
If the check fails, the server closes the SMTP conversation before receiving the mail. This avoids wasting resources.
The original qmail-smtpd program accepts by all messages, checking later for the existence of the recipients. So, if the message is delivered to a non-existant recipient a lot of additional system work and network traffic is generated, with several expensive bouncing if the sender is a fake.
chkuser has been developed with the goal to improve the acceptance SMTP phase of qmail-smtpd.
qmail-smtpd patched with chkuser may check the existence of e-mail recipients immediately in the SMTP acceptance phase of a message and rejects instantly all the messages sent to unexisting users, thus avoiding additional traffic, backscatter, workload and messages bounced more times.
These goals are achieved by enquiring the existing vpopmail archives (each format is supported: cdb, MySQL, LDAP, etc.) by using standard vpopmail calls, or using customized chkuser routines.
chkuser 2.0 has detailed logging of accepted and refused recipients and senders, thus allowing a deeper analysis of "who's sending to whom". This can facilitate any further enhancements of anti-SPAM features.
N.B. Current qmail version works only with my vpopmail-mysql package!
You can get more details here: http://debian.iuculano.it
So, let's go, in less than ten minutes we can have a full performant mail server!
Just open your terminal and add my deb repository in your sources.list:
debian:~# vi /etc/apt/sources.list
Add these lines:
deb http://debian.iuculano.it/apt sarge main contrib non-free deb-src http://debian.iuculano.it/apt sarge main contrib non-free
and then you can update and install the package:
debian:~# apt-get update
My vpopmail package uses mysql backend, so we need mysql-server:
apt-get install mysql-server
It is strongly recommended that you set a password for the mysql root user (which is NOT the same as the "normal" root user) with the command:
mysqladmin -u root password 'enter-your-good-new-password-here'
Now we can create a database and a user which is allowed to access it:
debian:~# mysqladmin -u root -p create vpopmail
debian:~# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 10 to server version: 4.0.24_Debian-10sarge2-log
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql> GRANT ALL PRIVILEGES ON `vpopmail` . * TO 'vpopmail'@'localhost'
IDENTIFIED BY 'some_pass' WITH GRANT OPTION ;
Query OK, 0 rows affected (0.01 sec)
mysql> quit
Bye
debian:~#
Good, now we can install qmail, vpopmail, spamassasin:
debian:~# apt-get install qmail-src spamassassin vpopmail-mysql \ spamc razor pyzor ucspi-tcp-src libmailtools-perl \ libmail-spf-query-perl libsys-hostname-long-perl
We build ucspi-tcp:
debian:~# build-ucspi-tcp
And finally, we build qmail:
debian:~# build-qmail
Don't worry if you get an error message when installing the .deb package; we need to remove exim4!
debian:~# dpkg --force-depends --purge exim4 exim4-base exim4-config \ exim4-daemon-light
Now:
debian:~# dpkg -i /tmp/qmail/qmai*.deb
Very well, now if you want SMTP AUTH and chkuser, you should edit /etc/init.d/qmail.
If you want chkuser you should change CHKUSER_START to DOMAIN or ALWAYS
If you want chkuser you should replace:
with:-u `id -u qmaild` -g `id -g nobody` -x /etc/tcp.smtp.cdb 0 smtp \
-u vpopmail -g vckpw -x /etc/tcp.smtp.cdb 0 smtp \
Standard Qmail setup now follows. Setting your mail name:
vi /etc/qmail/me
And add your mail name, for example mail.domain.org
Since we're using MySQL we need to specify the username and password to connect to the database with for vpopmail:
debian:~# vi /etc/vpopmail/vpopmail.mysql
and restart qmail and popmail POP3
debian:~# /etc/init.d/qmail restart Stopping mail-transfer agent: qmail. Starting mail-transfer agent: qmail. debian:~# /etc/init.d/vpopmail-mysql restart Restarting vpopmail pop3 server: vpopmail. debian:~#
Very well, we're almost done!
Some check-ups:
debian:~# apt-get install recode [...] debian:~# vadddomain test.bogus Please enter password for postmaster: enter password again: debian:~# ls -la /var/lib/vpopmail/domains/ totale 3 drwx------ 3 vpopmail vchkpw 1024 2006-07-07 16:34 . drwxr-xr-x 6 root root 1024 2006-07-07 16:09 .. drwx------ 3 vpopmail vchkpw 1024 2006-07-07 16:34 test.bogus debian:~# echo -en "postmaster@test.bogus" | recode data..base64 cG9zdG1hc3RlckB0ZXN0LmJvZ3Vz debian:~# echo -en "test" | recode data..base64 dGVzdA== debian:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. 220 mail.domain.com ESMTP quit 221 mail.domain.com Connection closed by foreign host. debian:~# less /var/log/syslog debian:~# telnet localhost 25 Trying 127.0.0.1... Connected to localhost.localdomain. Escape character is '^]'. 220 mail.domain.com ESMTP HELO TEST 250 mail.domain.com EHLO 250-mail.domain.com 250-AUTH LOGIN CRAM-MD5 PLAIN 250-AUTH=LOGIN CRAM-MD5 PLAIN 250-PIPELINING 250 8BITMIME AUTH LOGIN 334 VXNlcm5hbWU6 cG9zdG1hc3RlckB0ZXN0LmJvZ3Vz 334 UGFzc3dvcmQ6 dGVzdA== 235 ok, postmaster@test.bogus, go ahead (#2.0.0) mail from: test@nonexact.from 511 sorry, can't find a valid MX for sender domain (#5.1.1 - chkuser) mail from: test@test.com 250 ok RCPT TO: notexist@test.bogus 511 sorry, no mailbox here by that name (#5.1.1 - chkuser) RCPT TO: postmaster@test.bogus 250 ok quit 221 mail.domain.com Connection closed by foreign host. debian:~#
Good! Everything seems to work. The last thing we want to do is add anti-spam and anti-virus support:
debian:~# apt-get install qmailadmin autorespond ezmlm-src \ clamav clamav-daemon clamav-freshclam ripmime
Install simscan (http://www.inter7.com/simscan/simscan-1.2.tar.gz) and edit init.d/qmail. Here is an example of configuring simscan:
./configure --enable-user=clamav --enable-clamav=y \ --enable-custom-smtp-reject=y --enable-attach=y --enable-spam=y \ --enable-spam-hits=14 --enable-spamc-user=y --enable-received=y \ --enable-clamavdb-path=/var/lib/clamav --enable-spam-auth-user=n \ --enable-quarantinedir=/var/qmail/quarantine --enable-dropmsg=yGiuseppe Iuculano
Managed AntiSpam
Fully managed filtering of your incoming email.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I had to remove the 4 in all the exim4 bits to uninstall it
sorry to waste your time
[ Parent | Reply to this comment ]
# ps uxaww |grep -i qmail
qmaill 2607 0.0 0.1 1368 288 ? S Jul21 0:00 /usr/bin/multilog t s10000000 n30 /var/log/qmail/send
qmails 20402 0.0 0.1 1536 368 pts/1 S 10:04 0:00 qmail-send
qmaill 20403 0.0 0.1 1500 444 pts/1 S 10:04 0:00 splogger qmail 2
root 20404 0.0 0.1 1500 316 pts/1 S 10:04 0:00 qmail-lspawn |/usr/sbin/qmail-procmail
qmailr 20408 0.0 0.1 1496 312 pts/1 S 10:04 0:00 qmail-rspawn
qmailq 20409 0.0 0.1 1488 320 pts/1 S 10:04 0:00 qmail-clean
And i have also other error :
# /etc/init.d/qmail restart
Stopping mail-transfer agent: qmail.
Starting mail-transfer agent: qmailtcpserver: usage: tcpserver [ -1UXpPhHrRoOdDqQv ] [ -c limit ] [ -x rules.cdb ] [ -B banner ] [ -g gid ] [ -u uid ] [ -b backlog ] [ -l localname ] [ -t timeout ] host port program
# /etc/init.d/vpopmail-mysql restart
Restarting vpopmail pop3 server: /etc/init.d/vpopmail-mysql: line 83: cd: /var/lib/svscan: No such file or directory
If you have an idea... :)
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I remove it and try again :
# apt-get remove daemontools
# /etc/init.d/qmail restart
Stopping mail-transfer agent: qmail.
Starting mail-transfer agent: qmailtcpserver: usage: tcpserver [ -1UXpPhHrRoOdDqQv ] [ -c limit ] [ -x rules.cdb ] [ -B banner ] [ -g gid ] [ -u uid ] [ -b backlog ] [ -l localname ] [ -t timeout ] host port program
# /etc/init.d/vpopmail-mysql restart
Restarting vpopmail pop3 server: vpopmail.
telnet localhost 25
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
PS : excuse me for my bad english... i'm french
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Now :
/etc/init.d/qmail restart
Stopping mail-transfer agent: qmail.
Starting mail-transfer agent: qmail.
/etc/init.d/vpopmail-mysql restart
Restarting vpopmail pop3 server: vpopmail.
telnet localhost 25
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
GRRRrrrr !
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Jul 26 10:04:21 localhost qmail: 1153901061.597772 status: local 0/10 remote 0/20
Jul 26 10:28:25 localhost qmail: 1153902505.118105 status: exiting
Jul 26 10:28:25 localhost qmail: 1153902505.181246 status: local 0/10 remote 0/20
Jul 26 11:18:45 localhost qmail: 1153905525.509018 status: exiting
Jul 26 11:18:45 localhost qmail: 1153905525.548116 status: local 0/10 remote 0/20
Jul 26 14:06:46 localhost qmail: 1153915606.618625 status: exiting
Jul 26 14:06:46 localhost qmail: 1153915606.656347 status: local 0/10 remote 0/20
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
qmails 25133 0.0 0.1 1536 368 pts/0 S 14:06 0:00 qmail-send
qmaill 25134 0.0 0.1 1500 444 pts/0 S 14:06 0:00 splogger qmail 2
root 25135 0.0 0.1 1500 316 pts/0 S 14:06 0:00 qmail-lspawn |/usr/sbin/qmail-procmail
qmailr 25139 0.0 0.1 1496 312 pts/0 S 14:06 0:00 qmail-rspawn
qmailq 25140 0.0 0.1 1488 320 pts/0 S 14:06 0:00 qmail-clean
root 25193 0.0 0.2 2072 764 pts/0 R+ 14:41 0:00 grep -i qmail
and
netstat -tuapn | grep -i 25
tcp6 0 52 ::ffff:xxx.xx.xx.xx:22 ::ffff:xxx.xx.xx.xx:4275 ESTABLISHED24975/0
I have done various testing installation before using your howto, if you don't have idea of my problem i think i could do a format an done a new clean instalation...
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I do this a new time : build-ucspi-tcp
And i let the default folder and option during installation
Enter a directory where you would like to do this [/tmp/ucspi-tcp] /tmp/ucspi-tcp
All seems great :
/etc/init.d/qmail restart
Stopping mail-transfer agent: qmail.
Starting mail-transfer agent: qmail.
/etc/init.d/vpopmail-mysql restart
Restarting vpopmail pop3 server: vpopmail.
but...
telnet localhost 25
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I have really screwed up my system running on Debian. Before I used to have have stable in my sources.list. However I later change it to testing. After having tried the above mentioned article. I get the following errors towards the end of: build-qmail:
./compile chkuser.c
./load qmail-smtpd chkuser.o qregex.o rcpthosts.o commands.o timeoutread.o \
timeoutwrite.o ip.o ipme.o ipalloc.o control.o constmap.o \
received.o date822fmt.o now.o qmail.o dns.o cdb.a fd.a wait.a \
datetime.a getln.a open.a sig.a case.a env.a stralloc.a \
alloc.a strerr.a substdio.a error.a str.a fs.a auto_qmail.o base64.o `head -1 /var/lib/vpopmail/etc/lib_deps` `cat dns.lib` `cat \
socket.lib` `cat dns.lib`
head: cannot open `/var/lib/vpopmail/etc/lib_deps' for reading: No such file or directory
chkuser.o: In function `chkuser_cleanup':
chkuser.c:(.text+0x1f7): undefined reference to `vclose'
chkuser.o: In function `chkuser_realrcpt':
chkuser.c:(.text+0x10db): undefined reference to `vget_assign'
chkuser.c:(.text+0x11d6): undefined reference to `valias_select'
chkuser.c:(.text+0x12ec): undefined reference to `vauth_getpw'
chkuser.c:(.text+0x1392): undefined reference to `format_maildirquota'
chkuser.c:(.text+0x13a3): undefined reference to `vmaildir_readquota'
collect2: ld returned 1 exit status
make[1]: *** [qmail-smtpd] Error 1
make[1]: Leaving directory `/tmp/qmail/qmail-1.03'
make: *** [build] Error 2
I have really put my system in a mess regarding this. I have tried uninstalling and installing it again but in vain.
Any help would really be appreciated.
Thanks in advance,
Tek Bahadur Limbu
[ Parent | Reply to this comment ]
I was using qmail for eight years, with several patches, with vpopmail and mysql and... some day I've had to learn exim4 (with postgresql). Believe me, I didn't expect such a flexibility. Every single feature or behaviour I missed when I was using qmail I can now easily implement by making not too complex changes in exim config and I still have most of mail routing and/or mailbox info in database. It is really next generation smtp engine.
Cheers,
R.
[ Parent | Reply to this comment ]
I didn't use this guide, and instead compiled courier by hand to support qmails auth module and everything is working fine.
[ Parent | Reply to this comment ]
1. Qmail is unable to reject undeliverable email during the incoming SMTP conversation. It swallows everything and bounces it later. Over 90% of email today is spam or malware and almost all of it gives false envelope-sender and From: addresses. If the domains don't resolve or they get deferred, all this junk piles up in your queue. When the pile gets big enough, a denial of service occurs due to a known bug: qmail-send spends all its time scanning and sorting the queue and little mail goes out. I've seen this happen on a *big* machine (3 GHz Xeon with 4 GB memory) with less than twenty thousand messages in its queue. When the false addresses are deliverable, you batter some innocent victim with "backscatter." If you backscatter AOL and Yahoo, they'll block you. Been there, done that, got the tee shirt.
2. Qmail initiates a new SMTP conversation for each recipient. If you have 500 Yahoo addresses on a Mailman list, Qmail will connect to Yahoo and send the same exact message 500 times. An MTA designed for modern times (Postfix, Sendmail, Exim...) will connect once, present the 500 addresses, and send the message once. That's amazingly faster if you're used to Qmail's performance with mailing lists. And if the message matches Yahoo's content filtering for spam, which happens all the time with legitimate traffic, it scores 500 times instead of once and you get blocked.
Both of these problems are architectural. They can't be fixed without patching Qmail so extensively that it isn't Qmail any more.
3. Problem with qmail-command. It is up to the user who creates a set of dot-qmail files to avoid forwarding loops. Qmail-local does not generate its "Delivered-To:" line for pipe deliveries. A user can create a loop where two aliases forward to one another. This can take over the machine and make it hard to stop. But even worse, if a message with a Cc: list gets stuck in there, you mailbomb the Cc: address recipients. Been there, done that. So think twice before you let users create their own dot-qmail files.
4. You can't support well-known domains without extensive source IP blocking. There is way too much junk to filter it after you receive it. I used rblsmtpd and built a large tcprules file, covering most of the world's large consumer ISPs' dynamics and major spammers. When the dot-cdb file reached about 10 MB, it started having both false negatives and false positives. (Hash collisions? I didn't try to figure out what its problem was, but I had the same problem on woody, sarge, and Mandriva.) Spam got through that shouldn't, folks got blocked for no good reason. I switched to rbldnsd.
You may get away with running Qmail for a personal hobby server. But don't try to support dozens of domains with Mailman lists with thousands of subscribers. There's no reason when MTAs without these problems are available in Debian stable/main.
5. Alias files for Exim, Sendmail, and Postfix are fairly similar. Simple ones are compatible without change. But when your Qmail installation runs out of gas, which it will, switching from dot-qmail files and /var/qmail/control/* to the standard structure is a major pain.
Cameron
[ Parent | Reply to this comment ]
Jul 19 06:43:28 localhost qmail: 1153284208.178909 CHKUSER rejected rcpt: from <matilda@ringdahl.com::> remote <tta.ao2ie.org:unknown:59.95 .248.193> rcpt <guest@xxxx.com> : not existing recipient
2) Sometimes some big mail server reject your email if you have more than 'x' RCPT TO in a single connection (worm and spammer protection), so what is the best choice? I prefer qmail approch.
3)We are in 2006, you should not create dot-qmail files from console, you should use some nice web interface, (qmailadmin for example). Qmailadmin is very stable, no loop and no problems.
4) I don't understand your problem, rbl lists work good in my experience, perhaps you should not use aggressive lists.
I use:
sbl-xbl.spamhaus.org korea.services.net list.dsbl.org relays.ordb.org with rblsmtpd.
and:
dul.dnsbl.sorbs.net combined.njabl.org in dnsbllist (qmail-dnsbl patch).
With this configuration, I haven't any false, and dynamic IP clients can authenticate on qmail skipping rbl check.
I have qmail for big domains and my clients are happy and have no problems.
The only thing that was wrong with Qmail at the time was the stupid license.
The author no longer maintains Qmail. Other people publish patches, but because of the stupid license it isn’t possible to distribute binaries of the patched versions.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I followed the steps all the way and got stuck on testing the domain part. Does anyone know why I get:
server1:/etc/vpopmail# vadddomain test.bogus
Error - no authentication database connection. Initial open.
Thanks.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
but i must use socket . then what to do?
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
/etc/init.d/qmail start produces following errors:
mail.info:
Dec 17 14:39:30 asterix qmail: 1166362770.832731 tcpserver: fatal: unable to bin
d: address already used
Dec 17 14:39:30 asterix qmail: 1166362770.862761 alert: cannot start: unable to
switch to queue directory
So i guess i have made a small typo or something somewhere, only problem is, i dont know where... Help :(
/etc/init.d/qmail:
http://asterix.serveftp.org/~arcanus/qmail.init.txt
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
build-qmail # Say 'No' to Wanna install? and Wanna remove source?
cd /tmp/qmail/qmail-1.03
make clean
- fix the permissions in /etc/passwd and /etc/group according to INSTALL.ids (where the problem was i guess, wrong gids and uids...)
make setup check
./config
echo host.whatever > /var/qmail/control/me
echo "*@host.whatever" >> /var/qmail/control/rcpthosts
===
Now it works :D
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
A couple of probs:
1.) When I debian~# /etc/init.d/qmail restart
I get:
/etc/init.d/qmail: line 48: -u: command not found
/etc/init.d/qmail: line 48: -u: command not found
Stopping mail-transfer agent: qmail
/etc/init.d/qmail: line 48: -u: command not found
Starting mail-transfer agent: qmail
debian~# /etc/init.d/vpopmail-mysql restart
Restarting vpopmail pop3 server: vpopmail
2.) Proceeding to "Some check-ups:" in above tutorial
Everything OK until I get to: AUTH LOGIN
334 VXNlcm5hbWU6
check-up stalls at this point
Any suggestions? Thanks.
[ Parent | Reply to this comment ]
You can try this line
debian:~# echo -en "your-postmaster-password" | recode data..base64
instead of
debian:~# echo -en "test" | recode data..base64
and use the result for the probe suggested in the tutorial
I hope this helps
[ Parent | Reply to this comment ]
Thank you.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
/etc/init.d/qmail restart
I keep getting an error:
Starting mail-transfer agent: qmailid: vchkpw: No such user
I've been looking around and I didn't find any reference to the user vchkpw, only the vpopmail user and vchkpw group. Does anyone has an ideia of what could be going on?
[ Parent | Reply to this comment ]