Weblogs for mar

Posted by mar on Tue 25 Aug 2009 at 11:58
Tags: , , ,

Intro

Being a long time user of pdnsd (small and simple DNS caching daemon), I have always struggled a bit when using it with dhclient.

I have always used some custom hook scripts (pre-up, post-up, ...) in /etc/network/interfaces when I was configuring a device with dhcp to supply the just generated /etc/resolv.conf to pdnsd and generate a new one with nameserver 127.0.0.1 line in it to enable local caching. It got a bit more complicated with shorewall. Anyway, it has been a somewhat working setup, even with mapping stanzas in /etc/network/interfaces.

Before going on, I should mention I just hate the resolvconf debian package, which never seems to be doing what I want. (You may give it a try yourself.)

I was reading through the dhclient man page in hope there are some hooks or something and to my surprise, I learned they are! So finally, the much improved "receipt" for happy networking with laptop with several virtual machines, local DNS zones, local DNS caching and more ... ;-).

So much for the intro. Now the thing.

Setup

  • pdnsd should be able to use the info provided by dhclient, thankfully, it has a feature for this. Apart from other standard server stanzas, it should contain this server section. It points to file with standard resolv.conf syntax and it could be for example resolv.conf file generated by dhclient.
server {
    label = "resolv.conf.from.dhclient";
    file = "/etc/resolv.conf.from.dhclient";
}
  • Debian has another nice feature -- /etc/dhcp3/dhclient-enter-hooks.d/ and /etc/dhcp3/dhclient-exit-hooks.d/ directories, that can contain various scripts that are run before and after the dhclient obtains its info from server (and writes down the new /etc/resolv.conf). All i have to do is to prepare 2 simple custom scripts for pdnsd. This scripts are sourced ('.') by shell during startup/rebound/stop/... actions of dhclient.

/etc/dhcp3/dhclient-enter-hooks.d/dhclient-pdnsd-enter-hook This script just uses predefined variables by /sbin/dhclient-script and modifies them to its needs -- particullary the $new_domain_name_servers var:

# place a link to this file in /etc/dhcp3/dhclient-enter-hooks.d

echo "I: prepending 127.0.0.1 as local caching DNS server"
new_domain_name_servers="127.0.0.1 $new_domain_name_servers"
echo "I: new_domain_name_servers=$new_domain_name_servers"

and

/etc/dhcp3/dhclient-exit-hooks.d/dhclient-pdnsd-exit-hook This script generates the file mentioned in the server section of pdnsd above and notifies pdnsd that the config changed and it should re-read the DNS info.

# place a link to this file in /etc/dhcp3/dhclient-exit-hooks.d

pdnsd_resolv_conf="/etc/resolve.conf.from.dhclient"
echo "I: creating $pdnsd_resolv_conf"
cat /etc/resolv.conf | fgrep --invert-match -e "127.0.0.1" > $pdnsd_resolv_conf
echo "I: notifying pdnsd"
/usr/sbin/pdnsd-ctl config

Outro

With a setup like this, which integrates nicely into debian ways I can use pdnsd, which i like, for a long time, without worrying that future DHCP REBOUND will overwrite my /etc/resolv.conf leaving out my custom changes to this file -- ie. my local cache and local zones for virtual machines defined in pdnsd.

This set up also integrates with /etc/network/interfaces in a completely transparent way -- on a laptop, when you may be connected via bluetooth, wlan, eth, ppp -- this always ensures, that once DHCP is used, local DNS set up will nicely prevail no matter what interface is in use.

I would like to hear from you, how you handle DNS on your laptop, what tools you use and why. Until then, another happy day with debian, i must do ifdown wlan0 just to see it working again :).

 

Posted by mar on Sat 6 Jun 2009 at 15:48
Tags: , ,

Summary

Recently I was forced to compute mail bandwith per domain. Our MTA is Postfix, reporting tool is Awstats. After some googling I found some hints but they have to be slightly modified to present agregated per domain statistics.

Resources and software

  1. Awstats -- especially the FAQ: http://awstats.sourceforge.net/docs/awstats_faq.html#MAIL
  2. Prepflog -- no debian package: http://web.tiscali.it/postfix/prepflog.html
  3. Postfix of course

Comments on debian setup

These comments are not step by step how-to; just hints to someone like me, who tries to force Awstat show per domain mail statistics.

Software

First thing to note -- Debian installs awstats scripts into /usr/share/doc/awstats/examples. Beacause of this, you have some trouble to generate the default Awstats config, but you dont have to do that -- you can simply copy the default and modify it (dont forget to remove/modify the Include directive if you copy the default one).

Prepflog is probably a good idea, though i just believe to what is said on the product page -- it should narrow the log, so no duplicate entries are present.

Principle

You use your standard /var/log/mail.info.0 Postfix log, you pipe it through several usefull perl one liners producing CLF (common logging format) file for Awstats. Thankfully, Awstats is capable of proccessing a piped input directly.

Settings

Generate and modify the config for awstats

Create a config according to
http://awstats.sourceforge.ne/docs/awstats_faq.html#MAIL

It is important to set all the options as required -- with two notable exceptions -- LogFile and SiteDomain. For example my config looks like:

LogFile="/root/hampejz/awstats_mail_filter_by_domain.sh |"
SiteDomain="domain.mail.example.com"
Include "/etc/awstats/awstats.0.mail.example.com.common"

Note that LogFile points to my custom filter. Include just includes standard Awstats file according to URL above.

Prepare the custom filter

#! /bin/sh

# filters the postfix mail log to be used by awstats
# if the generation takes long, switch off DNS lookup in awstats config        
# filtering is preproccessed by perl to collect per-domain statistics

cat /var/log/mail.info.0 | \
    perl /root/hampejz/prepflog.pl | \
    perl /usr/share/doc/awstats/examples/maillogconvert.pl standard | \
    perl -pe 's/<>/user\@unknown/g' | \
    perl -pe 's/(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} )((\S+)(@\S+ ))((\S+)(@\S+ ))/\1\4\7/g' | \
    perl -pe 's/(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} )((\S+)(@\S+ ))((\S+ ))/\1\4\@localhost /g'

Some comments:

  1. We use /var/log/mail.info.0 -- my system is configured with delayed compression of logfiles -- so I can always use static log files to generate statistics (they are modified no more by daemons).
  2. We run the prepflog (I just believe the agenda :)
  3. We run the Awstats filter, which produces the CFL file. From now on, the Awstats is capable of processing the result, though it would show the mail by user, not by domain.
  4. We handle the mail with no sender (some kind of spam and some locally generated messages from ugly crontabs)
  5. 's/(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} )((\S+)(@\S+ ))((\S+)(@\S+ ))/\1\4\7/g' -- regexp guru could make it nicer, but I just want to make it readable. This changes

    DATE TIME sender@somewhere1 recepient@somewhere2 ANYTHING

    into

    DATE TIME @somewhere1 @somewhere2 ANYTHING.

    This leaves all the mail from sending and receiving domain to be agregated (it is taken as a mail from single user).
  6. 's/(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} )((\S+)(@\S+ ))((\S+ ))/\1\4\@localhost /g'

    handles locally generated mail for local delivery (ie. no domain) as a mail for @localhost.

Crontab

All of this should be run from crontab. I believe you can handle that, so let it be the homework for a reader -- my file in /etc/cron.hourly:


#! /bin/sh

# updates awstats files

files=`ls /etc/awstats/awstats.*.conf`

for i in $files; do
    domain=`echo $i | sed -e 's/\/etc\/awstats\/awstats\.\(.*\)\.conf/\1/g'`
    # generate statistics and build static pages in one command
    mkdir -p /var/www/awstats/$domain
    perl /usr/share/doc/awstats/examples/awstats_buildstaticpages.pl \
	-config=$domain \
        -configdir=/etc/awstats \
        -update \
        -output \
        -staticlinks \
        -awstatsprog=/usr/lib/cgi-bin/awstats.pl \
        -dir=/var/www/awstats/$domain \
        -diricons=/awstats-icon
    chown www-data.www-data -R /var/www/awstats/$domain
done
chown www-data.www-data /var/lib/awstats/*

 

Posted by mar on Sat 18 Oct 2008 at 18:21

Debian apt repositories use digital keys to sign packages. See http://wiki.debian.org/SecureApt for nasty details, but you may find yourself downloading a software (ie. emacs-snapshot) from semi-official site and you may want to add that repository to sources.list. In that case, a good idea is also to add the distributor's key to apt. Quick guide follows (run as root):

1. See what keys you already trust:

apt-key list

2. Download the key you want to add (keys are stored in files similar to SSL certificates).

3. Add the key to apt:

apt-key add /path/to/downloaded/key_file.pgp

4. Check that the key is now added -- see ad 1.

Of course, you should do this only with keys that you trust, but that is another story.

 

Posted by mar on Sat 3 Feb 2007 at 19:06
Tags: none.

Just a note how to generate and install the SSL self-signed certificate under JBoss 3.2.8 AS.

This entry has been truncated read the full entry.

 

Posted by mar on Wed 31 Jan 2007 at 09:10
Tags: ,

2009-03-10 update: for udevtest and udevinfo commands use udevadm test and udevadm info in newer Linux distributions.

Recently I was forced to edit some udev init scripts to properly assign device names under the /dev. I will show how to assign the same names to your devices under the /dev, so whenever you plug-in your camera you will find the symlink /dev/my_camera pointing to the real device, whatever it may be. Hope you will find this mini-how-to helpfull.

This entry has been truncated read the full entry.