Debian Administration

Creating logfile archives with logrotate

Many of the services installed on Linux machines will produce logfiles which grow, and grow, and grow. If left unchecked you can easily fill a disk with a large collection of logfiles if you're not careful.

The most common method of keeping logfile growth in check is to use logrotate, and many Debian packages are setup to work with this by default.

The most obvious package which uses it is Apache, the webserver, which by default keeps its logfiles in the directory /var/log/apache (or /var/log/apache2).

If you examine this directory you will see that there are a bunch of logfiles which are archived:

root@mystery:~# ls -1 /var/log/apache2/
access.log
access.log.1
access.log.2.gz
access.log.3.gz
access.log.4.gz
access.log.5.gz
error.log
error.log.1
error.log.2.gz
error.log.3.gz
error.log.4.gz
error.log.5.gz

Here the current logfiles access.log, error.log are kept raw as are yesterday's logfiles (access.log.1 and error.log.1). Previous logfiles are compressed with gzip and only kept for five weeks. (I know it's five weeks and not five days because I've looked at the configuration - It's not clear from this output though!)

The process that is in charge of compressing and rotating these logfiles is called logrotate and it is executed once per day upon Debian installations.

As we saw when we were looking at scheduling commands with cron there is a directory called /etc/cron.daily which contains scripts which are executed once per day. Here you will find the logrotate driver script.

Every day this script runs and examines two things:

• The configuration file /etc/logrotate.conf
• The configuration directory /etc/logrotate.d

The latter is where most of our packages are configured. This directory contains configuration files which other packages have installed. For example if you install apache the file /etc/logrotate.d/apache will be installed.

Many servers such as exim the mailserver will install their own configuration file, and you can add your own.

A typical logrotate configuration file looks like this:

/var/log/apache/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if [ -f /var/run/apache.pid ]; then
                        /etc/init.d/apache restart > /dev/null
                fi
        endscript
}

You can see several important things here. The most obvious is the list of files that will be matched by this configuration file:

/var/log/apache/*.log {
...
}

After this we have a collection of configuration terms, a different one on each line. In the example above we have:

• weekly
  • The files should be rotated every week. Opposite: daily
• rotate nn
  • We should keep no more than nn files.
• compress
  • Compress older files with gzip. Opposite: nocompress
• delaycompress
  • Don't compress yesterdays files. Opposite: compress
• notifempty
  • Don't do any rotation if the logfile is empty. Opposite: ifempty
• create xx user group
  • If we have to create the new file give it the given mode, owner, and group.
• sharedscripts
  • Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts.
• postrotate + endscript
  • Anything between these is executed after the rotation process. Opposite : prerotate

Hopefully that should have made sense!

The upshot of this script is that any file which matches /var/log/apache/*.log is rotated every week, compressed, if it's non-empty. The new file is created with the file mode of 640, and after the rotation has finished the server is restarted.

If we wish to install a local service which creates a logfile we can cause it to be rotated very easily, just by adding a new logrotate configuration file.

Assuming we have a new service "fred" which produces its output in /var/log/fred/output.log we can cause this to be rotated every day with a script like this:

/var/log/fred/*.log {
  daily
  missingok
  rotate 7
  compress
  delaycompress
  create 640 fred fred
  sharedscripts
     /etc/init.d/fred restart
  endscript
}

This will:

• Run daily.
• Keep no more than 7 days worth of logfiles at any one time.
• Not complain if there is a logfile missing.
• Compress the older files, but not yesterdays.
• Create the new logfiles as being owned by the user and group fred.
• Restart the service after rotating the logfiles.

Any of the existing files in the logrotate directory can be examined for more examples - and the manpage documents all the options you may use in a clear manner:

man logrotate

Add Comment

 

<<< Working with Debian binary packages Replacing binaries with dpkg-divert >>>