Avoiding slow package updates with package diffs
Posted by Steve on Thu 14 Sep 2006 at 08:56
If you're using the unstable or testing distribution of Debian GNU/Linux you will almost certainly have noticed that apt-get uses daily-diffs for its package updates. In many common situtations this is more bandwidth efficient, however it isn't always appropriate.
apt-get is a standard command which is used by many Debian users to manage package installation, and upgrades. (Although there are also other package managers such as synaptic, or aptitude.)
Typically apt-get works in a two stage process:
- "apt-get update"
The update sub-command instructs apt-get to connect to list of package sources and download a list of all the available packages to the local system. These lists contain the names, descriptions, version information and dependency information of all the packages available for installation.
Using these lists apt-get can be instructed to install a package from a remote source, upgrade all packages, search for Debian packages, and conduct other operations.
- "apt-get upgrade
The upgrade command is a familiar one to most users, it instructs the apt-get tool to install any available package upgrades which are available from the remote site(s) you're using.
This command will not install packages which aren't currently installed in your system - instead it will inform you that some packages have been kept back.
(There are also other sub-commands available to apt-get; see the manpage for details by running "man apt-get".)
Until recently each time you ran "apt-get update" you would download each complete package list from the sources you've got configured - regardless of the fact that very little might have changed since the previous time you did so.
As an example here is part of the output from my desktop system when I run this command:
subliminal@messages:~# apt-get update .. .. Get: 5 http://ftp.uk.debian.org sid/main Packages [4354kB] Get: 6 http://ftp.uk.debian.org sid/contrib Packages [62.1kB] Get: 7 http://ftp.uk.debian.org sid/non-free Packages [87.3kB] Get: 8 http://ftp.uk.debian.org sid/main Sources [1233kB] Get: 9 http://ftp.uk.debian.org sid/contrib Sources [21.2kB] Get: 10 http://ftp.uk.debian.org sid/non-free Sources [28.5kB] Fetched 5792kB in 34s (168kB/s)
Here we can see that I've downloaded several files in the process, and the total size was 5792kB - or 5Mb.
In an attempt to make package updates more dial-up friendly, or easier for other users who are on a low-bandwidth link this system of package downloads was recently augmented by the addition of "package diffs". Rather than downloading the complete package lists upon each update only the difference to the package lists are fetched.
A package update could now look something like this:
notice@me:~# apt-get update ... ... Get: 27 2006-09-10-1306.48.pdiff [293B] Get: 28 2006-09-10-1306.48.pdiff [301B] Get: 29 2006-09-10-1306.48.pdiff [13.2kB] Get: 30 2006-09-11-1318.15.pdiff [60.1kB] Get: 31 2006-09-11-1318.15.pdiff [60.1kB] Get: 32 2006-09-11-1318.15.pdiff [436B] Get: 33 2006-09-11-1318.15.pdiff [436B] Get: 34 2006-09-11-1318.15.pdiff [257B] Get: 35 2006-09-11-1318.15.pdiff [257B] Get: 36 2006-09-12-1306.11.pdiff [136B] Get: 37 2006-09-10-1306.48.pdiff [271B] Get: 38 2006-09-11-1318.15.pdiff [60.1kB] Get: 39 2006-09-11-1318.15.pdiff [11.0kB] Get: 40 2006-09-11-1318.15.pdiff [11.0kB] Get: 41 2006-09-11-1318.15.pdiff [230B] Get: 42 2006-09-11-1318.15.pdiff [230B] Get: 43 2006-09-11-1318.15.pdiff [436B] Get: 44 2006-09-11-1318.15.pdiff [257B] Get: 45 2006-09-11-1318.15.pdiff [11.0kB] Get: 46 2006-09-12-1306.11.pdiff [41.9kB] Get: 47 2006-09-12-1306.11.pdiff [41.9kB] Get: 48 2006-09-12-1306.11.pdiff [439B] Get: 49 2006-09-12-1306.11.pdiff [439B] Get: 50 2006-09-11-1318.15.pdiff [230B] Get: 51 2006-09-12-1306.11.pdiff [41.9kB] Get: 52 2006-09-12-1306.11.pdiff [9377B] Get: 53 2006-09-12-1306.11.pdiff [9377B] Get: 54 2006-09-12-1306.11.pdiff [439B] Get: 55 2006-09-12-1306.11.pdiff [9377B] Fetched 396kB in 38s (10.4kB/s)
This time we see that the total download was only 396kB - a significant saving! However the downside is apparent, instead of downloading several files we've downloaded over fifty and even though we've downloaded less data the time taken to do so was significantly longer.
If you don't frequently update your system then it may make more sense for you to revert to the old "inefficient" system and thankfully this is a simple process.
Rather than running "apt-get update" you can run:
apt-get update -o Acquire::Pdiffs=false
You can even choose to make this behaviour the default by adding editing the file /etc/apt/apt.conf (creating it if it is not present). Simply add the following line: