Setting Up Awstats for Apache

Posted by k2 on Mon 24 Oct 2005 at 00:05

I have noticed people asking questions about web server statistics programs on debian-user mailing list. So I'm posting a compendious version of my original(personal) documentation on how to setup awstats on Debian GNU/Linux for generating statistics for Apache web server. The article focuses on the steps required to get awstats running on your server, setting up cron job for it, and reconfiguring logrotate for apache to take care of lost logs.

The first step is to install awstats using apt or dselect or your favourite package management software. With apt just execute (as root):

~# apt-get install awstats

Run the awstats configure script to setup awstats for the domain you want to monitor. Remember to repeat the configuration process for each domain you want to monitor.

~# cd /usr/share/doc/awstats/examples/
~#/usr/share/doc/awstats/examples$ perl awstats_configure.pl

Answer the questions asked by the script in which you will need to know the full configuration file path of your web server(e.g. /etc/apache/httpd.conf), and the name of your domain for which you are setting up awstats( full-domain-name in this article).

While running the script it will complain that it is not being run from the default location (/usr/local/awstats). Answer yes to the prompt to run from the non-standard location as the official Debian package of awstats places the script in /usr/share/doc/awstats/examples/ which is the Debian way.(See the original for detailed script output)

After the script exits, you should set up an alias for awstats-icons in httpd.conf. Inserted it in the Aliases section. The awstats_configure.pl script adds it in the end of httpd.conf - comment/remove it.

Your entry for /awstats-icon/ in the Aliases section should look like:

Alias /awstats-icon/ /usr/share/awstats/icon/
<Directory /usr/share/awstats/icon>
  Options None
  AllowOverride None
  Order allow,deny
  Allow from all
</Directory>

Now you can restart apache:

~# /etc/init.d/apache restart

Edit the awstats' domain configuration file you just created /etc/awstats/awstats.full-domain-name.conf to have LogFormat=1. Default "LogFormat=4" does not show Browsers, OS, keywords, etc. Do make sure that the parameter SiteDomain has a value of the domain to monitor. Generate stats the first time by using the following command. (You can allow awstats to read your apache log file by changing permissions. Run "chmod o+r /var/log/apache/access.log"):

~#/usr/lib/cgi-bin/awstats.pl -config=awstats.full-domain-name.conf

Update for config "/etc/awstats/awstats.full-domain-name.conf"
With data in log file "/var/log/apache/access.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 60
Found 0 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 60 new qualified records.

You can now check the stats in your browser by point it to http://full-domain-name/cgi-bin/awstats.pl?config=full-domain-name.

Set up cron job for awstats as below:

3,33 * * * * www-data [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=full-domain-name -update >/dev/null

Setup a prerotate task to run awstats.pl before log rotate and change permissions of log file to 644. Please comment on this as making access.log 644 is basically allowing access to anybody to read the file. Is there another way to do it more securely?

/var/log/apache/*.log {
  weekly
  missingok
  rotate 52
  compress
  delaycompress
  notifempty
  create 644 root adm
  sharedscripts
  prerotate
    /usr/lib/cgi-bin/awstats.pl -config=ghaint.no-ip.org -update
  endscript
  postrotate
  if [ -f /var/run/apache.pid ]; then \
    if [ -x /usr/sbin/invoke-rc.d ]; then \
      invoke-rc.d apache reload > /dev/null; \
    else \
      /etc/init.d/apache reload > /dev/null; \
    fi; \
  fi;
  endscript
}

This should get awstats running automatically every half an hour with the statistics available via your web browser. After finishing you might want to go back to the configuration file /etc/awstats/awstats.full-domain-name.conf to do more tweaking. Reading the documentation in /usr/share/doc/awstats/ is also recommended.

 

 


Posted by simonw (84.45.xx.xx) on Mon 24 Oct 2005 at 01:12
[ View Weblogs ]
Can I ask the meta-question - what stats do people find useful?

I use "visitors" (apt-get install visitors ; visitors -A -o text /path/to/file/in/apache/combined/log/format), and have been talking to author (and Debian maintainer) about various minor issues which have been fixed. Be interested in how they compare, awstats looks more comprehensive and prettier, but I've seen bad security press - is that when run as a CGI?

Whilst I find browser share, and OS share interesting, I think the "404" report (because it shows broken links), and the Google keywords (because it reveals spelling mistakes, more than what people search for).

Also I am mostly after a regular report for a quick overview, than being routinely updated (I can always run the command line again to find out what happened in the last N minutes), since I need it pushed to me for me to even bother looking.

Also interested in what people send to their customers.

[ Parent | Reply to this comment ]

Posted by Anonymous (213.164.xx.xx) on Mon 24 Oct 2005 at 10:53
I personally have never found a statistics package that does what I want.
visitors with graphwiz output has come close though.

If you have a small site, you want to see where people come from, and what route they take through your site, what things interest them.
If you have a large site, aggregrate data is more useful because analysing each visitors routes would take forever to read.

[ Parent | Reply to this comment ]

Posted by mahdi (217.113.xx.xx) on Mon 24 Oct 2005 at 12:54
Yep - last time awstats have had some security problems, but it's kind of service that doesn't need to be publicly available. Here's simple method that I'm using to restrict access to it:
<VirtualHost *>
...
<Directory "/usr/lib/cgi-bin">
    AllowOverride None
    Options ExecCGI -MultiViews +SymLinksIfOwnerMatch
    Order allow,deny
    Allow from all

    AuthType basic
    AuthName "cgi-bin restricted"
    AuthUserFile /etc/awstats/awstats.htpasswd

    <Files awstats.pl>
        Require valid-user
    </Files>
</Directory>
</VirtualHost>
Now it's only left to define valid users by: htpasswd /etc/awstats/awstats.htpasswd username

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Mon 24 Oct 2005 at 13:16
[ View Steve's Scratchpad | View Weblogs ]

To be honest I make the statistics available for my websites mostly because I can, rather than because I actually need to.

I find it interesting to watch the visitor count change - for example this site's stats show a growing readership month-on-month since it went live.

Other than that I don't really pay much attention. It is nice to see referers (sic) and search strings, but I can typically notice a sudden influx of readers by other means. (Either manually tailing acess.log - which I do a lot - or watching the performance drop!)

Seeing 404s can be useful, but mostly I rely upon peoples ability to search if they go wrong - it is the 500 ("internal server error") which I pay most attention to of all the error codes.

And to answer your question all of the Awstats security issues to date have involved it running as a CGI script.

Steve
--

[ Parent | Reply to this comment ]

Posted by Anonymous (213.164.xx.xx) on Mon 24 Oct 2005 at 18:28
It would be good if you masked your visitors' hostnames and ip addresses in the public stats page in the same way as you mask them on d-a.org

[ Parent | Reply to this comment ]

Posted by Anonymous (128.131.xx.xx) on Wed 26 Oct 2005 at 00:05
indeed shows a lot of referrer spam too.

the user agent detection of webalizer is too old for todays client.
beside it's age webalizer has still an quick and nice overview. webalizer is easier to configure than modlogan. (never figured out there how i could display with modlogan say instead of the top 30 pages the the top 100 pages).
still all those are not as powerfull as one would wish - like showing the pathes visitors took, nor any statistical anomaly..

[ Parent | Reply to this comment ]

Posted by suspended user gg234 (195.14.xx.xx) on Thu 27 Oct 2005 at 14:51
i have some virtual hosts and how to configure each use can see their stats please help me with this where i can define all the users lists each user is having different directory.

[ Parent | Reply to this comment ]

Posted by Anonymous (210.1.xx.xx) on Tue 28 Oct 2008 at 11:29
Try this. Create a log dir in each domain. Add this line (without quotes): "CustomLog /var/www/mysite/log/access.log combined" to your sites-enabled/000-default or whatever your distro apache conf requires.
e.g.
<VirtualHost *>
DocumentRoot /var/www/mysite
ServerName www.mysite.com
ServerAlias mysite.com
CustomLog /var/www/mysite/log/access.log combined
<IfModule mod_php5.c>
php_value include_path ".:/var/www/mysite/includes"
</IfModule>
</VirtualHost>

hth
build

[ Parent | Reply to this comment ]

Posted by Arto (213.250.xx.xx) on Fri 4 Nov 2005 at 17:23
I use Visitors as well, as it's written in C it's pretty darned fast compared to most of the Perl-based log analyzers out there. Not to mention the fact that it nicely adheres to the KISS paradigm so it's exceedingly easy to setup and use.

I run a number of websites and mostly I'm interested in the daily and weekly unique visitors, and where they came from (referrer URLs, search engine statistics). Couldn't care less about the number of hits, page loads and so on.

Because a certain number of stats will won't be accurately recorded on the server end due to proxies, I also use services like StatCounter.com to complement the Visitors stats. StatCounter uses a small, invisible bit of JavaScript in the footer of web pages to do its tracking, meaning it'll track unique visitors with close 100% accuracy (for those that have JavaScript disabled, it still counts them using a transparent 1x1 pixel image), proxies be darned. It can also track information such as screen resolution, which simply isn't available in server-side logs.

[ Parent | Reply to this comment ]

Posted by Anonymous (65.42.xx.xx) on Fri 28 Oct 2005 at 10:54
I'm running Debian 3.1 and installed the latest awstats in the stable branch.

Why do I get awstats-6.4-1 instead of awstats-6.4-1.1 which contains the 2-month old fix for a command-injection security vulnerability?

I get all the security updates for other packages like Apache2 perfectly fine--but not awstats.

Any ideas?

Oh, I've been running the stable branch in hopes that an "apt-get update && apt-get upgrade" cron job would keep me up-to-date with the latest security patches without worrying about compatibility. Last year, I really didn't like tracking the testing branch because security updates took forever--I thought I was escaping that by going with stable...

[ Parent | Reply to this comment ]

Posted by k2 (69.157.xx.xx) on Sat 29 Oct 2005 at 04:36
[ View Weblogs ]
The latest version for awstats for stable version is 6.4-1 as shown by the package's overview page. You could still force apt to install a particular version(if you want to) which is not there in stable yet.

[ Parent | Reply to this comment ]

Posted by Anonymous (64.108.xx.xx) on Sun 30 Oct 2005 at 04:10
Two questions:

1. Why is 6.4-1 the latest version for stable, when 6.4-1.1 fixed a serious security vulnerability 2 months ago?

2. Why are sysadmin's forced to use non-stable branch versions in order to get security fixes?

Note: I'm not inquiring about 6.4-2, I'm inquiring about 6.4-1.1 which is the reason stated (as being already fixed) for closing tickets complaining about the lack of security fix in stable branch.

[ Parent | Reply to this comment ]

Posted by Anonymous (141.211.xx.xx) on Wed 16 Nov 2005 at 22:17
1. Why is 6.4-1 the latest version for stable, when 6.4-1.1 fixed a serious security vulnerability 2 months ago?

Because the functionality of 6.4-1 has been tested with the rest of the stable branch and has been verified as being "stable." The security hole has been patched though. That's what the -1 means - one Debian update beyond the official 6.4 version.

2. Why are sysadmin's forced to use non-stable branch versions in order to get security fixes?

They aren't. All security patches are backported to the stable version, usually sooner than they are applied to testing & unstable versions. The version number doesn't change, because the software version hasn't changed.

[ Parent | Reply to this comment ]

Posted by Anonymous (213.44.xx.xx) on Thu 3 Nov 2005 at 14:32
Give 'Go Awstats' a try. It automates the production of Awstats Apache log reports at a site where multiple virtual hosts exist. It always updates the reports for the current month and the current year and only produces other reports if they do not exist, which saves on CPU. The regeneration of a report can be forced by simply erasing it.

http://www.ruwenzori.net/code/go_awstats/

[ Parent | Reply to this comment ]

Posted by Anonymous (142.163.xx.xx) on Sat 19 Nov 2005 at 22:07
I'm getting this error when I run awstats_configure.pl:
-----> Create config file '/etc/awstats/awstats.vh1.cjb.net.conf'
Error: Failed to open '/usr/share/doc/awstats/examples/wwwroot/cgi-bin/awstats.model.co nf' for read.

[ Parent | Reply to this comment ]

Posted by k2 (69.157.xx.xx) on Sat 19 Nov 2005 at 23:13
[ View Weblogs ]

In the link, also provided above, for the full detailed article; I mentioned that inspite of the error in the last step, the configuration file was created in the desired location. Check if it was created for you or not.

[ Parent | Reply to this comment ]

Posted by Anonymous (142.163.xx.xx) on Sun 20 Nov 2005 at 10:20
it would be in /etc/awstats, right?
the only two files there are awstats.conf and awstats.conf.local.
the profile name I entered was 'vh1.cjb.net'

[ Parent | Reply to this comment ]

Posted by k2 (69.157.xx.xx) on Sun 20 Nov 2005 at 16:11
[ View Weblogs ]

Yes, it should be there. But you could always make it yourselves too. Copy the default awstats.conf to awstats.vh1.cjb.net.conf and edit it to fit your needs.

[ Parent | Reply to this comment ]

Posted by Anonymous (81.231.xx.xx) on Thu 16 Feb 2006 at 11:16
Well, same problem here on my side; the requesting file doesn't exist and the path is wrong. Don't know, but correct path would be /usr/share/doc/awstats/examples but this doesn't solve the missing file awstats.model.conf.

[ Parent | Reply to this comment ]

Posted by Anonymous (81.231.xx.xx) on Thu 16 Feb 2006 at 11:22
And as well, when trying to run the script local using following:

/usr/lib/cgi-bin/awstats.pl -config=awstats.www.site.net.conf -update

I get following error:

Error: SiteDomain parameter not defined in your config/domain file. You must edit it for using this version of AWStats.
Setup ('/etc/awstats/awstats.conf' file, web server or permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs' directory).

But, the SiteDomain parameter has been defined within the used conf file which is located in /etc/awstats and as well the awstats.pl is executed from the same directory.

[ Parent | Reply to this comment ]

Posted by Anonymous (68.124.xx.xx) on Tue 29 Aug 2006 at 22:34
I am having the exact same problem. Does anyone understand what's happening?

[ Parent | Reply to this comment ]

Posted by rodonnell (86.42.xx.xx) on Mon 31 Mar 2008 at 18:56

I ran into this too, and eventually figured out that awstats was reading the generic file awstats.conf - which of course had -

SiteDomain = ""

- and this was despite the -conf option telling it to read the site-specific file. When I renamed awstats.conf to something else, it then read the file specified by -conf.

The thing I found by trial and (lots of) error was that I had to omit both the "awstats." prefix and ".conf" suffix from the file name in the -conf option - in other words, the command was as follows-

/usr/lib/cgi-bin/awstats.pl -config=www.site.net -update

and NOT ... -config=awstats.www.site.net.conf ....

I hope this helps someone.

[ Parent | Reply to this comment ]

Posted by Anonymous (212.37.xx.xx) on Tue 30 Sep 2008 at 13:45
It certainly did.

[ Parent | Reply to this comment ]

Posted by Anonymous (69.243.xx.xx) on Wed 7 Apr 2010 at 02:00
Yeeeeah! it works! kudos. Im sorry this is really exciting for me lol.

[ Parent | Reply to this comment ]

Posted by Anonymous (67.201.xx.xx) on Mon 7 Mar 2011 at 07:30
THANK YOU!

[ Parent | Reply to this comment ]

Posted by Anonymous (80.136.xx.xx) on Thu 5 Oct 2006 at 15:54
I experienced the same trouble. The mentioned file obviously doesn't exist. To cope with that, make sure you unzipped the awstats.model.conf file:

  gunzip /usr/share/doc/awstats/examples/awstats.model.conf.gz

Then, edit

/usr/share/doc/awstats/examples/awstats_configure.pl

where you will find some important parameters near head of file, which are meant to be edited by package maintainers. Unfortunately this seems not to have happened for Debian. But you can do it yourself.

For my purposes the following values worked:

$AWSTATS_PATH='/usr/share/awstats';
$AWSTATS_ICON_PATH='/usr/share/awstats/icon';
$AWSTATS_CSS_PATH='/usr/share/awstats/css';
$AWSTATS_CLASSES_PATH='/usr/share/awstats/lib';
$AWSTATS_CGI_PATH='/usr/lib/cgi-bin';
$AWSTATS_MODEL_CONFIG='/usr/share/doc/awstats/examples/awstats.mo del.conf';
$AWSTATS_DIRDATA_PATH='/var/lib/awstats';


Please note, that you also may have to modify these paths in the generated apache.conf manually afterwards.

[ Parent | Reply to this comment ]

Posted by Override (195.234.xx.xx) on Mon 16 May 2011 at 07:29
For some reason this awstats.model.conf was gunzipped yet.

After an gunzip -r /usr/share/doc/awstats/examples/wwwroot/cgi-bin/awstats.model.con f.gz everything worked fine...

[ Parent | Reply to this comment ]

Posted by Override (195.234.xx.xx) on Mon 16 May 2011 at 07:39
Oops.. I meant offcourse:
gunzip -r /usr/share/doc/awstats/examples/awstats.model.conf.gz

[ Parent | Reply to this comment ]

Posted by Antras (85.206.xx.xx) on Sun 30 Apr 2006 at 10:15
Can you advise any analyse-tool, which save its result not at the html-files, but at the database(MySQL e.g.)? Webalyser&AWstats can't works with database :(
I've found only sawmill, but it's shareware.

[ Parent | Reply to this comment ]

Posted by Anonymous (59.164.xx.xx) on Sat 26 Jun 2010 at 04:37
I have setup awstats but the browser link is not working.

http://serverip/cgi-bin/awstats.pl?config=domainname

The link returns that itis broken.

What am I doing wrong?



[ Parent | Reply to this comment ]

Posted by Anonymous (131.111.xx.xx) on Wed 2 Nov 2011 at 09:26
I couldn't get the cron configuration in the article to work. The one in the AWStats documentation worked for me, after altering the path to awstats.pl for the Debian environment. This calculates the stats at 03:50 every day:
50 3 * * * /usr/lib/cgi-bin/awstats.pl -config=www.mysite.com -update >/dev/null

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 471 votes ~ 5 comments )

 

 

Related Links