Disabling Apache logging of particular requests

Posted by Steve on Thu 7 Aug 2014 at 10:26

I've got a webserver which runs a particular service, and that service is sufficiently interesting and important that I run monitoring probes against it every 30 seconds. Those particular requests clutter my logs, and here is a brief note on how to avoid that clutter.

If you're using Apache you'll be used to code such as the following to enable logging:

#
# Log all accesses:
#
CustomLog /var/log/apache2/access.log combined

Some people prefer to have dated logfiles, without the need for log-rotation. They'll be configured like this:

#
# Log to a per-day file, but ensure there is always a symlink to the
# most recent/current logfile:
#
CustomLog "|/usr/bin/cronolog --symlink=/var/log/apache2/current.log /var/log/apache2/access.log_%Y_%m_%d" combined

(This assumes the package is installed, which is achieved via "apt-get install cronolog".)

In either case we can change this to disable logging of particular requests by appending "env=!dontlog", giving us a line like this:

CustomLog /var/log/apache2/access.log combined env=!dontlog

Or:

CustomLog "|/usr/bin/cronolog --symlink=/var/log/apache2/current.log /var/log/apache2/access.log_%Y_%m_%d" combined env=!dontlog

This update ensures that logging will be skipped if the "env" contains the flag "dontlog". How is that set? Via mod_env.

First of all you'll need to ensure that is enabled, by running:

# a2enmod env

Once the module is enabled you can update your virtual host to disable logging for a particular user-agent:

# Don't log requests made by curl
SetEnvIFNoCase User-Agent "curl/" dontlog

Or, ideally, you can avoid logging by a particular remote IP:

SetEnvIf Remote_Addr "1.2.3.4" dontlog

Once you've enabled the module, added the exclusion-argument appropriately you can restart Apache to make the change take effect:

# /etc/init.d/apache2 restart

 

Nginx

For reference nginx allows something similar - This minimal example disables the logging of requests to http://example.com/check:

location = /check {
   # ...
   access_log off;
   # ...
}

 

 

 


Posted by Anonymous (189.180.xx.xx) on Thu 7 Aug 2014 at 14:15
Right. But as you are opening a pipe to an external program, it could even work this way:
CustomLog "|grep -v '^192\.168\.0\.[[:digit]]+.*GET /admin_or_cron' | \
/usr/bin/cronolog --symlink=/var/log/apache2/current.log /var/log/apache2/access.log_%Y_%m_%d" combined

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Thu 7 Aug 2014 at 14:18
[ View Steve's Scratchpad | View Weblogs ]

The biggest issue with your approach is that you exclude a matching line regardless of where it matched.

Imagine you wanted to avoid logging based on source-IP matching 10.20.30.40 as you seem to be doing in your example - then consider a request which would be logged like this:

1.2.3.4 GET /foo/bar?ip=10.20.30.40 HTTP/1.0 ...

The actual IP making the request was 1.2.3.4, but because of the parameter .. well it isn't logged and you'd not expect that.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (89.71.xx.xx) on Thu 7 Aug 2014 at 17:45
Except for the anchor a the beginning of the regexp.

[ Parent | Reply to this comment ]

Posted by Steve (2.126.xx.xx) on Thu 7 Aug 2014 at 17:48
[ View Steve's Scratchpad | View Weblogs ]

D'oh!

Otherwise it's a valid point relating to blocking on request, or user-agent though.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (71.10.xx.xx) on Sat 9 Aug 2014 at 01:08
I wouldnt do this grep since its not easily maintained and doesnt scale well.

If I did I would line buffer the grep however (-line-buffered or stdbuf -o0 grep)

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 475 votes ~ 5 comments )