Weblog entry #366 for simonw
See all weblog entries made by simonw.
#366
Dear Lazyweb - Apache Log and slow start
Posted by simonw on Mon 8 Mar 2010 at 10:41
Problems restarting Apache on one server.
Receiving a huge amount of traffic due to some Chinese click jacker who can't spell their own domain name on one virtualhost. When Apache restarts it compiles some Perl for FastCGI which causes a delay (and fcgid hits its process limits till the compiling is all done - which is as expected for a busy FastCGI and not usually an issue). But this morning it was all too much (it was running some other stuff as well) the slow response caused Apache to hit MaxClients, and it sat swamped responding to requests very slowly.
The huge Chinese traffic was just getting a 301 redirect. I stopped that virtual server and it all started normally. Since the new error response was larger than the 301, it occurred to me that the change was really to log less. So I disabled logging on the virtual host getting the spurious traffic from China, and suddenly the box has enough horse power to deal with the requests.
I didn't see obvious evidence of disk I/O problems. So anyone see a bottleneck on Apache log files before? Hostname look-up is disabled. The server does hold open a few hundred log files, which is probably not ideal. Didn't see any file handle warnings. The URLs logged are chunky, 3 or 4 line (200+ characters between GET and PROTOCOL).
Failing that anyone who can make money out of lots of horny Chinese people, you can be the destination of that 301 redirect for a small donation ;)
On the other hand this is the first non-IE6 traffic from this click-jacker so presumably he "fixed" his Javascript to work with modern browsers :(
Receiving a huge amount of traffic due to some Chinese click jacker who can't spell their own domain name on one virtualhost. When Apache restarts it compiles some Perl for FastCGI which causes a delay (and fcgid hits its process limits till the compiling is all done - which is as expected for a busy FastCGI and not usually an issue). But this morning it was all too much (it was running some other stuff as well) the slow response caused Apache to hit MaxClients, and it sat swamped responding to requests very slowly.
The huge Chinese traffic was just getting a 301 redirect. I stopped that virtual server and it all started normally. Since the new error response was larger than the 301, it occurred to me that the change was really to log less. So I disabled logging on the virtual host getting the spurious traffic from China, and suddenly the box has enough horse power to deal with the requests.
I didn't see obvious evidence of disk I/O problems. So anyone see a bottleneck on Apache log files before? Hostname look-up is disabled. The server does hold open a few hundred log files, which is probably not ideal. Didn't see any file handle warnings. The URLs logged are chunky, 3 or 4 line (200+ characters between GET and PROTOCOL).
Failing that anyone who can make money out of lots of horny Chinese people, you can be the destination of that 301 redirect for a small donation ;)
On the other hand this is the first non-IE6 traffic from this click-jacker so presumably he "fixed" his Javascript to work with modern browsers :(