Creating logfile archives with logrotate

Posted by Steve on Tue 29 Mar 2005 at 06:11

Tags:

Many of the services installed on Linux machines will produce logfiles which grow, and grow, and grow. If left unchecked you can easily fill a disk with a large collection of logfiles if you're not careful.

The most common method of keeping logfile growth in check is to use logrotate, and many Debian packages are setup to work with this by default.

The most obvious package which uses it is Apache, the webserver, which by default keeps its logfiles in the directory /var/log/apache (or /var/log/apache2).

If you examine this directory you will see that there are a bunch of logfiles which are archived:

root@mystery:~# ls -1 /var/log/apache2/
access.log
access.log.1
access.log.2.gz
access.log.3.gz
access.log.4.gz
access.log.5.gz
error.log
error.log.1
error.log.2.gz
error.log.3.gz
error.log.4.gz
error.log.5.gz

Here the current logfiles access.log, error.log are kept raw as are yesterday's logfiles (access.log.1 and error.log.1). Previous logfiles are compressed with gzip and only kept for five weeks. (I know it's five weeks and not five days because I've looked at the configuration - It's not clear from this output though!)

The process that is in charge of compressing and rotating these logfiles is called logrotate and it is executed once per day upon Debian installations.

As we saw when we were looking at scheduling commands with cron there is a directory called /etc/cron.daily which contains scripts which are executed once per day. Here you will find the logrotate driver script.

Every day this script runs and examines two things:

  • The configuration file /etc/logrotate.conf
  • The configuration directory /etc/logrotate.d

The latter is where most of our packages are configured. This directory contains configuration files which other packages have installed. For example if you install apache the file /etc/logrotate.d/apache will be installed.

Many servers such as exim the mailserver will install their own configuration file, and you can add your own.

A typical logrotate configuration file looks like this:

/var/log/apache/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
                if [ -f /var/run/apache.pid ]; then
                        /etc/init.d/apache restart > /dev/null
                fi
        endscript
}

You can see several important things here. The most obvious is the list of files that will be matched by this configuration file:

/var/log/apache/*.log {
...
}

After this we have a collection of configuration terms, a different one on each line. In the example above we have:

  • weekly
    • The files should be rotated every week. Opposite: daily
  • rotate nn
    • We should keep no more than nn files.
  • compress
    • Compress older files with gzip. Opposite: nocompress
  • delaycompress
    • Don't compress yesterdays files. Opposite: compress
  • notifempty
    • Don't do any rotation if the logfile is empty. Opposite: ifempty
  • create xx user group
    • If we have to create the new file give it the given mode, owner, and group.
  • sharedscripts
    • Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts.
  • postrotate + endscript
    • Anything between these is executed after the rotation process. Opposite : prerotate

Hopefully that should have made sense!

The upshot of this script is that any file which matches /var/log/apache/*.log is rotated every week, compressed, if it's non-empty. The new file is created with the file mode of 640, and after the rotation has finished the server is restarted.

If we wish to install a local service which creates a logfile we can cause it to be rotated very easily, just by adding a new logrotate configuration file.

Assuming we have a new service "fred" which produces its output in /var/log/fred/output.log we can cause this to be rotated every day with a script like this:

/var/log/fred/*.log {
  daily
  missingok
  rotate 7
  compress
  delaycompress
  create 640 fred fred
  sharedscripts
     /etc/init.d/fred restart
  endscript
}

This will:

  • Run daily.
  • Keep no more than 7 days worth of logfiles at any one time.
  • Not complain if there is a logfile missing.
  • Compress the older files, but not yesterdays.
  • Create the new logfiles as being owned by the user and group fred.
  • Restart the service after rotating the logfiles.

Any of the existing files in the logrotate directory can be examined for more examples - and the manpage documents all the options you may use in a clear manner:

man logrotate

 

 


Posted by Anonymous (83.165.xx.xx) on Wed 30 Mar 2005 at 21:34
Hi!

I want my file /var/log/auth.log to be in "append only" mode (chattr +a auth.log). Would the following be an appropiate section for logrotate.conf to achieve my purpose?

/var/log/auth.log {
weekly
rotate 4
prerotate
chattr -a /var/log/auth.log
endscript
postrotate
chattr +a /var/log/auth.log
killall -HUP syslogd
endscript
}

Thanks!

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Wed 30 Mar 2005 at 21:46
[ View Steve's Scratchpad | View Weblogs ]

That looks good - if you run the following command you'll get to see what will happen for your given file:

logrotate --debug --force nameofyourfile

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (83.165.xx.xx) on Thu 31 Mar 2005 at 21:04
Yep! Yesterday it seemed to work in debug mode, and today it perfectly rotated the log and keeped the append mode on.

Thank you!

[ Parent | Reply to this comment ]

Posted by figjam (210.54.xx.xx) on Fri 1 Apr 2005 at 07:14
[ View figjam's Scratchpad | View Weblogs ]
Hi,

What about if I have a directory where there are multiple files per day? I would like to create a daily archive file, keeping seven days and do something like the delaycompress option to make sure I don't compress a log file currently being written to.

There is a file every 15 minutes that looks something like 200504011800.poller, 200504011815.poller and so on.

Cheers,

[ Parent | Reply to this comment ]

Posted by chrisk1981 (83.65.xx.xx) on Mon 4 Apr 2005 at 10:50
One problem I encountered was that logs can only be rotated on the same filesystem (anybody know why?)

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Sat 20 Aug 2005 at 13:25
[ View Steve's Scratchpad | View Weblogs ]

That is an implementation artifact, caused by the code using the rename function to deal with the files.

(I guess because this is simpler to code and avoids the common error associated with "copying and then renaming": a disk being full.)

I too get annoyed by this, and usually "fix" it by doing a move myself in the postrotate section of the script.

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (206.176.xx.xx) on Mon 25 Jul 2005 at 23:43
I could use some help:

I'm running p0f as root and it writes its output to /var/log/p0f.log. I have p0f.log in my logrotate.conf but when logrotate is run via cron, a new p0f file gets created but nothing gets written to it. It stays empty until I kill the process and re-run it from the command line. What I am doing wrong?

Here's the listing of my logrotate.conf file:

/var/log/p0f.log {
rotate 7
daily
delaycompress
missingok
create 0666 root root
}

I tried changing the create mask and the user and group but it doesn't seem to make a difference. Should I try cron as the owner?

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Mon 25 Jul 2005 at 23:48
[ View Steve's Scratchpad | View Weblogs ]

You probably want to take advantage of the pre/post-rotate actiosn to restart p0f, it might be expecting to use the old file still.

Otherwise it looks good. If you save the snippet into /etc/logrotate.d/p0f you can use the following command line to debug the issue in either case:

logrotate -d -f /etc/logrotate.d/p0f

That might be useful ..?

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by kuriharu (64.142.xx.xx) on Tue 26 Jul 2005 at 04:53
Thanks for the tip. I copied that info from my /etc/logrotate.conf file and created a file in /etc/logrotate.c/p0f. Looking in /etc/logrotate.d I see a lot of files that restart themselves differently using postrotate. Is there a generic way to restart something like p0f? It resides in /usr/sbin, I think.

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Tue 26 Jul 2005 at 10:32
[ View Steve's Scratchpad | View Weblogs ]

Usually most long running processes, and daemons are started via start-stop-daemon - have a look at the scripts in /etc/init.d for example uses.

This records the process ID into a file, and allows it to be killed cleanly later.

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by kuriharu (206.176.xx.xx) on Tue 26 Jul 2005 at 15:23
I found how to get p0f to terminate and how to restart it (it was easy, I used 'killall', which I just learned about).

It works great -- thanks for all your help. My p0f files are getting rotated daily.

Long live Debian! :)

[ Parent | Reply to this comment ]

Posted by Anonymous (173.9.xx.xx) on Wed 10 Feb 2010 at 00:42
I believe that logrotate does not work well unless files are opened in append mode (rhel-as4). In my case, when opened 'trunc' the service writing to the log would keep a file pointer and start writing at the same offset as before the rotation, and the preceeding text would be replaced by garbage. Closing and reopening, restarting the service or opening the file with append (or with >>&) all seem to work.

In my case, I could not restart the task as it was a manager for a piece of hdwr. Append worked for me.

[ Parent | Reply to this comment ]

Posted by Anonymous (89.166.xx.xx) on Tue 4 Oct 2011 at 20:28
Hey, i have had the same problem. After some googling and trying i found the answer:

You easily need to add "copytruncate" to the settings, thats all.

Background: The Problem is, that after logrotate rotated the logfile, the pointer still is on the file - but now it is named as *.gz and apache tries to log into that file - not into the newly created one.

website that helped me out: www.thegeekstuff.com/2010/07/logrotate-examples/
(watch out for point 3.)

- Domger

[ Parent | Reply to this comment ]

Posted by gmicah (141.211.xx.xx) on Mon 22 Aug 2005 at 12:28
I am setting up logrotation on some syslog messages that I am recieving from different boxes. What would be the easiest way to rotate logs recursively, all the directories within a directory?

[ Parent | Reply to this comment ]

Posted by suspended user gg234 (195.14.xx.xx) on Mon 17 Oct 2005 at 11:26
Hi,

I want to configure my logrotate configuration file as follows

we need keep all clients monthly logs to /var/log/apache2/logs folder
Under this folder we have different clients directories.Each directory
should contain the logfiles that is access.log and error.log files for
each folder.Every month end logrotate should run and it should keep the
log files in monthnameandyear folder with monthnameandyear.tgz file.

How to configure this in logrotate.conf

Example:-

logfiles location /var/log/apache2/logs under this folder every client is having their folder for logs
in side that apache log files are there i.e error.log,access.log.

Every month logrotate should run and it should create a folder with monthnameandyear under this monthname folder
that perticular month logfiles .tgz file shouls be there.

ex:- /var/log/apache2/logs/bahu/may2005/may2005.log.tgz

[ Parent | Reply to this comment ]

Posted by Anonymous (200.24.xx.xx) on Fri 8 Sep 2006 at 04:10
I'm looking for a way (or manner --sorry, my english is not so good--) to mail the rotated log as a attachment, not in the body of the message.
The fact is, i'm trying to rotate not a log, but a Openoffice file for backup reasons (this could be a non-common activity, but i'd like to do it). Unfortunately, logrotate's mail option sends the file in plain text on the body of the message. I'd like to send it like an attached file.

Thanks for your help.

Jader

[ Parent | Reply to this comment ]

Posted by Anonymous (24.160.xx.xx) on Fri 10 Nov 2006 at 02:48

[ Parent | Reply to this comment ]

Posted by butch (84.41.xx.xx) on Sun 31 Dec 2006 at 18:44
I hope someone can help me with the following.

I am using virtual apache2 hosts each with their own access log files. I have created seperate logrotate files for each host in /etc/logrotate.d/

What I experience is that my log files are being rotated but Apache keeps loging in the old logfile access.log.1 as if apache is not restarted. However I have

postrotate
if [ -f /var/run/apache2.pid ]; then
/etc/init.d/apache2 restart > /dev/null
fi
endscript


in each script.

Does anyone has a clue on how to get Apache loging into the new access.log file?

[ Parent | Reply to this comment ]

Posted by ComradeP (85.234.xx.xx) on Sun 8 Apr 2007 at 10:03
Did you resolve that problem in the meantime? I have exactly the same issue, and have to manually restart apache again (after logrotate has done) to get it to log to the right file.

I could be wrong, but I *think* this has only been happening since I introduced a potentially rather expensive pre-rotate task (awstats.pl)...

[ Parent | Reply to this comment ]

Posted by butch (84.41.xx.xx) on Sun 8 Apr 2007 at 10:16
No I did not resolve the problem. I have no clue on where to start.

I use awstats also, so this could be the cause of it.

[ Parent | Reply to this comment ]

Posted by ComradeP (85.234.xx.xx) on Thu 12 Apr 2007 at 16:58
Hi again,

I noticed something that was definitely wrong with my configuration, and that (conceivably) is to blame for the problem we're seeing: In adding the awstats 'prerotate' section, I forgot to close it off with 'endscript' (the pattern is 'prerotate {commands} endscript postrotate {commands} endscript').

Thus, upon careful perusal of the mails cron was sending me, I noticed it was complaining about 'postrotate: command not found' -- since there was no endscript after the prerotate, it continued executing each line as a prerotate command until the postrotate's 'endscript', which, as it happens, was the bit that restarts apache.

Now if this restart happens before the log rotation, it's not surprising that Apache keeps using the old file.

I'm not actually sure this will resolve the issue (I do weekly rotation and am somewhat reluctant to trigger one by hand), but I'm hopeful. :)

- P

[ Parent | Reply to this comment ]

Posted by butch (84.41.xx.xx) on Fri 13 Apr 2007 at 08:47
So if I understand you correctly you invoke awstats from the cron file for Apache's log rotation. Do you mind posting the contents of this cron job here?

My feeling is I use another setup. I have a logrotate cron job for every virtual host that logs its own access log file. Awstats is doing updates in a separate cron job.

[ Parent | Reply to this comment ]

Posted by ComradeP (85.234.xx.xx) on Fri 13 Apr 2007 at 10:08
I also have a cron job for awstats separately. The issue is that when logrotate moves logfiles out of the way, the part of the logfiles that haven't been seen by awstats is lost; thus, in addition to the separate cron job for awstats, I also invoke awstats from logrotate.

The relevant file is /etc/logrotate.d/apache2:

/var/log/apache2/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 644 root adm
        sharedscripts
        prerotate
                [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache2/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update
        endscript
        postrotate
                if [ -f /var/run/apache2.pid ]; then
                        /etc/init.d/apache2 restart > /dev/null
                fi
        endscript
}
The command in prerotate is exactly what my awstats cron job runs every 10 minutes. The problem I described was that I was missing the 'endscript' after that command, so that it considered everything until the final 'endscript' as part of the prerotate handler.

If this doesn't look familiar, then maybe you do have a different problem.. could you post your apache logrotate file?

- P

[ Parent | Reply to this comment ]

Posted by butch (84.41.xx.xx) on Fri 13 Apr 2007 at 13:32
Strange, I don't have an entry for awstats in my Apache logrotate file and it looks like we have the same problem.

For every virtual domain on my web server I use the following logrotate file in /etc/logrotate.d/virtualhostname:
/var/log/apache2/virtualhostname/*.log {
        weekly
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 644 root adm
        sharedscripts
        postrotate
                if [ -f /var/run/apache2.pid ]; then
                        /etc/init.d/apache2 restart > /dev/null
                fi
        endscript
}

[ Parent | Reply to this comment ]

Posted by ComradeP (85.234.xx.xx) on Fri 13 Apr 2007 at 14:04
Is it for all virtual hosts that Apache keeps logging in the rotated file, or just for some? The only potentially troublesome issue I see above is that apache will be restarted multiple times, once for each vhost... not that I see why that would make a difference.

Incidentally, you can use shell globs in the filename pattern for directories too -- so replacing all the different logrotate virtual-host-specific files with one that matches on /var/log/apache2/*/*.log would do the same, if I'm not mistaken.

I'll let you know if my "fix" has solved the issue for me.

- P

[ Parent | Reply to this comment ]

Posted by Anonymous (200.183.xx.xx) on Mon 12 May 2008 at 17:48
Your problem is that your apache is restarted after each logfile rotation.

I think you can solve this by using the following syntax:

/var/log/apache2/virtualhostname/*.log /var/log/apache2/virtualhostname2/*.log {
(...)
}

Regards,

Auro

[ Parent | Reply to this comment ]

Posted by Anonymous (68.230.xx.xx) on Wed 31 Jan 2007 at 18:15
daemon.log.*
kern.*
syslog.*
messages.*

These log files are being rotated, yet there is no mention of them in the /etc/logrotate.conf nor /etc/logrotate.d directory

[ Parent | Reply to this comment ]

Posted by Steve (62.30.xx.xx) on Wed 31 Jan 2007 at 22:27
[ View Steve's Scratchpad | View Weblogs ]

Assuming plain syslogd usage then via a special cronjob /etc/cron.daily/sysklogd.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (86.45.xx.xx) on Tue 16 Jun 2009 at 11:34
Why doesn't syslog use logrotate? Why does it do it itself?

[ Parent | Reply to this comment ]

Posted by mcs1 (134.148.xx.xx) on Mon 21 Apr 2008 at 05:03
Hi,

Unfortunately I have a big problem. Logrotate just does not rotate the syslog files. I have an up to date debian stable.

If I run logrotate -dvf /etc/logrotate.d/syslog-ng I get:


rotating pattern: /var/log/uucp.log  forced&n bsp;from command line (4 rotations)
empty log files are not rotated, ol d logs are removed
considering log /var/log/uucp.log
  log does not need rotating

rotating pattern: /var/log/syslog  forced&nbs p;from command line (7 rotations)
empty log files are rotated, old lo gs are removed
considering log /var/log/syslog
  log needs rotating
rotating log /var/log/syslog, log->rotateCount& nbsp;is 7
renaming /var/log/syslog.7.gz to /var/log/syslog.8 .gz (rotatecount 7, logstart 1, i 7 ),
renaming /var/log/syslog.6.gz to /var/log/syslog.7 .gz (rotatecount 7, logstart 1, i 6 ),
renaming /var/log/syslog.5.gz to /var/log/syslog.6 .gz (rotatecount 7, logstart 1, i 5 ),
renaming /var/log/syslog.4.gz to /var/log/syslog.5 .gz (rotatecount 7, logstart 1, i 4 ),
renaming /var/log/syslog.3.gz to /var/log/syslog.4 .gz (rotatecount 7, logstart 1, i 3 ),
renaming /var/log/syslog.2.gz to /var/log/syslog.3 .gz (rotatecount 7, logstart 1, i 2 ),
renaming /var/log/syslog.1.gz to /var/log/syslog.2 .gz (rotatecount 7, logstart 1, i 1 ),
renaming /var/log/syslog.0.gz to /var/log/syslog.1 .gz (rotatecount 7, logstart 1, i 0 ),
renaming /var/log/syslog to /var/log/syslog.1
running postrotate script
running script with arg /var/log/syslog:  ;"
      /etc/init.d/syslog-ng re load >/dev/null
"
error: unable to open /var/log/syslog.1  for compression


I have removed the other parts of the debug output, it is just the same.
The script /etc/logrotate.d/syslog-ng :

/var/log/messages {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}


/var/log/user.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}


/var/log/uucp.log {
   rotate 4
   missingok
   notifempty
   weekly
   compress
}

/var/log/syslog {
   rotate 7
   daily
   compress
   postrotate
      /etc/init.d/syslog-ng re load >/dev/null
   endscript
}


Again I have stripped a lot of the config.

It just does not rename the original file.
Any help is greatly appreciated!! I am fighting this for some time....

Thank you very much.

mcs1

[ Parent | Reply to this comment ]

Posted by mcs1 (134.148.xx.xx) on Mon 21 Apr 2008 at 06:26
Hi,

Sorry I have to get myself a big, big silly hat!
The problem lies in there:

logrotate -dvf /etc/logrotate.d/syslog-ng

-d debug means no changes are made!!! I am still waiting to see if it rotates with cron.
The original problem was with sysklogd, it did not want to logrotate with cron. Changed to syslog-ng, which has the logrotate setup that rotates /var/log/syslog, /var/log/auth.log etc. included in the scripts.
On another etch machine it works perfectly with sysklogd, even wqithout any additional scripts????
So still not at the bottom of this but at least it rotates now.
Thank you very much for a great resource!!
Mcs1

[ Parent | Reply to this comment ]

Posted by colinc (202.126.xx.xx) on Mon 25 Aug 2008 at 03:18
I would like to rotate our Apache log files. Starting with the following city-access_log.1 as it the main culprit filling up our var-disk-space. I would like to rotate the log file once a day and compress/gzip the file. Am tentative as had to use the graceful option to restart apache as not to disrupt web client access. Please advise if this can be done using the graceful option so nobody suffers any obvious outages

[ Parent | Reply to this comment ]

Posted by colinc (202.126.xx.xx) on Mon 25 Aug 2008 at 03:28
Here is my httpd file currently in /ec/logrotate.d

/var/log/httpd/*log {
missingok
notifempty
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}

which doesn't seem to be doing anything at present, but I cant see any seetings for weekly rotation, but I can see a kill command on the http pid which makes me worried I won't being performing a graceful restart of the apache provess.

[ Parent | Reply to this comment ]

Posted by colinc (202.126.xx.xx) on Tue 26 Aug 2008 at 00:47
ok, I'm answering my own question here, I guess I just replace the
/bin/kill -HUP `cat /usr/local/apache/logs/httpd.pid 2>/dev/null` 2> /dev/null ¦¦ true with /usr/sbin/apachectl graceful and it's job done. I will test using your suggestions above

[ Parent | Reply to this comment ]

Posted by Anonymous (68.236.xx.xx) on Tue 23 Sep 2008 at 16:08
Very helpful, thanks!

However, I think the description of the sharedscripts option is backwards:

"sharedscripts
Run any given prerotate or postrotate script for each logfile individually. Opposite: nosharedscripts. "

It's nosharedscripts that runs the scripts for each logfile individually. sharedscripts runs the scripts once, no matter how many logfiles are rotated.

[ Parent | Reply to this comment ]

Posted by Anonymous (202.54.xx.xx) on Thu 1 Mar 2012 at 10:33
I am not able to logrotate more than once .

below is my content of logrotate.conf file
/var/lib/helloworld/test-bin/staticvoicedevel/Logs/Hello/*.log {
daily
missingok
rotate 10
size 5k
create 0777 chakma chakma
}


[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 326 votes ~ 1 comments )