Speedup DNS requests with a local cache
Posted by Steve on Tue 25 Apr 2006 at 08:20
One common server bottleneck is DNS lookups. Many common server tasks such as from looking up hostnames to write Apache logfiles and processing incoming mail require the use of DNS queries. If you're running a high-traffic system it might be useful to cache previous lookups.
There are several different packages you can use for caching DNS requests - including bind, djbdns, dnsmasq and pdnsd.
The pdnsd package is a very simple and lightweight tool for DNS caching. It will, like many of the other systems, act as a small DNS server forwarding requests to a "real" DNS server and caching the responses.
When pdnsd is stopped it will save all the lookups which have been made against it so they may be reloaded when it starts again.
Installation is very straightforward:
apt-get install pdnsd
Once installed the software is configured via the file /etc/pdnsd.conf.
To configure the software you must do two things:
- Configure pdnsd so that it will forward requests it doesn't know about to a real DNS server, letting it cache those results.
- Update your system so that DNS lookups against the newly installed cache, or proxy.
Once you've completed these two steps all DNS lookups upon your system will be cached, and your DNS lookups should be much faster.
Upon your Debian GNU/Linux system you configure the DNS server(s) which are being used by means of the file, /etc/resolv.conf, this file will contain a list of name servers to query, perhaps along with a search domain to be used for unqualified hosts.
To tell your server to make DNS queries against the freshly installed server you would update that file to read:
nameserver 127.0.0.1
The next thing to do is to edit the pdnsd configuration file /etc/pdnsd.conf to specify which DNS servers the cache should use for its own lookups - these will most likely be your ISPs nameservers.
Locate the section of the configuration file which starts with server and add the IP address:
#
# Specify the IP address of the real DNS server to query against here:
#
server {
ip=11.22.33.44;
timeout=30;
interval=30;
uptest=ping;
ping_timeout=50;
purge_cache=off;
}
With this setting updated you can restart the caching service:
root@itchy:/etc# /etc/init.d/pdnsd restart Restarting proxy DNS server: pdnsd. root@itchy:/etc#
If you wish to add more DNS servers to query against you can add them seperated by commas, or you can add multiple ip= lines such as these two examples:
# Several IPs seperated by commas.
ip=11.22.33.44,111.222.333.444;
# Easier to read - one per line:
ip=11.22.33.44;
ip=111.222.333.444;
For more details of the supported options please consult the documentation by running "man pdnsd.conf".
You can test the cache is working by issuing a manual request to it:
root@itchy:/etc# dig @localhost example.com mx ;; QUESTION SECTION: ;example.com. IN MX ;; AUTHORITY SECTION: example.com. 86400 IN SOA dns1.icann.org. hostmaster.icann.org. ;; Query time: 2224 msec ;; SERVER: 192.168.1.50#53(192.168.1.50) ;; WHEN: Sun Apr 23 21:47:41 2006 ;; MSG SIZE rcvd: 90
Here we used the dig command (part of the dnsutils package) to lookup the MX record of the domain name example.com. Notice at the bottom it shows "Query time: 2224msec"? Lets run that same query again - if our cache is working correctly it should be significantly faster:
root@itchy:/etc# dig @itchy example.com mx |grep time ;; Query time: 1 msec
Much faster :)
(Yes DNS queries are ordinarily cached to a certain extent; so you'd expect the speedup even without our explicit DNS caching server...)
I've been using that setup quite happily for a while now -- I occasionally have some troubles with pdnsd not resuming properly when I suspend-to-disk, but they are quite easily fixed when they happen.
[ Parent | Reply to this comment ]
Is there really any reason to do this in real time, i.e. enabling HostnameLookups in the Apache configuration?
I mean, the DNS names are quite handy when creating webserver stats to guess where visitors are coming from etc., but most if not all statistics software for webserver logs can do DNS lookups. So where's the point in doing it in real time?
--
Debian GNU/Linux on an IBM Thinkpad T43p
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Honestly? I always disable real-time lookups.
I chose it as a reasonably common thing that other people frequently do ..
[ Parent | Reply to this comment ]
Otherwise, I have to agree on your other comments, some kind of DNS cache when using some slow link is really useful, even UMTS/3G connections (with its sometimes lousy latency, at least here in Austria) are improved a lot.
--
Debian GNU/Linux on an IBM Thinkpad T43p
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
I've given pdnsd a whirl a few times, but I was dissatisfied by its pipe backend, which I needed to use in order to establish horizons. It failed in a very sneaky way (offering up incorrect answers!), and I finally clocked enough hours tinkering with it to satisfy myself that it was time to go back to DJB's trusty dusty dnscache.
Installing dnscache (from djbdns) at least used to conflict with courier-mta due to both ucspi-tcp and courier-mta providing /usr/bin/addcr -- the easy way out is to compile ucspi-tcp from source so everything ends up in /usr/local/bin. The two versions of addcr seem to be compatible, so there's no harm done that I've detected in quite a long time of running both Courier and djbdns together.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
I always disable realtime lookups of Apache requests too, I just chose that as a common example.
I've noticed real appreciable speedups using the cache on dialup, and even over a slow ISP link - without any problems. I guess your mileage may vary, but it is a simple thing to setup and if it doesn't work out easy to reverse.
dnscache I've not tried, but I'm not a fan of the DJB-philosophy so that really is personal bias...
[ Parent | Reply to this comment ]
if you are running debian testing or unstable, another option is to use nscd with persistent databases something like this:
grabbed from Laptop LDAP
server-user nscd debug-level 0 reload-count unlimited paranoia no enable-cache passwd yes positive-time-to-live passwd 2592000 negative-time-to-live passwd 20 suggested-size passwd 211 check-files passwd yes persistent passwd yes shared passwd yes enable-cache group yes positive-time-to-live group 2592000 negative-time-to-live group 20 suggested-size group 211 check-files group yes persistent group yes shared group yes enable-cache hosts yes positive-time-to-live hosts 2592000 negative-time-to-live hosts 20 suggested-size hosts 211 check-files hosts yes persistent hosts yes shared hosts yes
[ Parent | Reply to this comment ]
I've gone down that PowerDNS trail three times now, and from scratch each time hoping to find that the pipe backend problems were my own mental misfiring. I'll probably try again later this year, and keep trying once or twice a year until the darn thing will work for me. Because I'm not a big DJB fan.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Just a quick comment to point out that pdnsd is not the same as PowerDNS - you might know that already, but it isn't 100% clear from your comment.
[ Parent | Reply to this comment ]
Head hung low in shame, I'm going to go find that caffeine I'm blaming this on.
[ Parent | Reply to this comment ]
I thought in Apache2 it was asynchronous, and as we use a separate DNS server, it is the cost of having the forked process open for the time it takes to do a DNS lookup (usually less than 2ms as it is usually cached).
I can see how a slashdotting might be a problem, but I'm curious to know exactly what the cost of this is. For most of my stuff it is enabled, as prompt reporting is a definite plus in my book, and I don't want to be shipping log files off to other machines for simple reporting.
[ Parent | Reply to this comment ]
it seems to work for me (dns-cache is ok after mta installation)
[ Parent | Reply to this comment ]
should of course read:Once installed the software is configured via the file /etc/dnsd.conf.
(extra p in front of dnsd.conf)Once installed the software is configured via the file /etc/pdnsd.conf.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Thanks, I've corrected this now.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
how to configure the software you must do two things:
* Configure pdnsd so that it will forward requests it doesn't know about to a real DNS server, letting it cache those results.
* Update your system so that DNS lookups against the newly installed cache, or proxy.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Your two points are addressed in the article:
- " Specify the IP address of the real DNS server to query against here:"
- "To tell your server to make DNS queries against the freshly installed server you would update that file to read:"
[ Parent | Reply to this comment ]
[ Send Message | View Utumno's Scratchpad | View Weblogs ]
Very simple package. 1 minute and everything is working.
One more advice for ppp/dsl users: you have to comment out option 'usepeerdns' in /etc/ppp/peers/[provider] so that your pppd will not overwrite /etc/resolv.conf with list of DNS servers it gets during the negotiation.
[ Parent | Reply to this comment ]