Rescuing systems using the Debian snapshot server
Posted by Steve on Mon 28 Aug 2006 at 08:46
One of the unofficial Debian project resources which doesn't get the attention it deserves is the Debian Snapshot site. The site contains a mirror of old Debian packages, which can be very useful for system recovery.
In most normal cases you won't ever need to use it, unless you're wanting to compare two different package versions to see changes, or do other non-standard things. However when you do need to use it you'll learn what a big lifesaver it is!
The biggest use for the site, for me, has been for recovering from broken package updates. Whilst these are rare in the Debian Stable and Testing releases they can be an issue when running Debian unstable.
As a cromulent example the Debian Xen packages available in unstable have recently become broken - they panic on boot. Reverting the package was difficult since the working copy had vanished from the Debian package repository, but the snapshot service allowed me to recover easily.
In my case this was the only option since the local package cache (in /var/cache/apt/archives) had been cleaned already - and the previous package was unavailable on the Debian mirror.
Thankfully kernel packages are nicely built such that you can have multiple versions installed simultaneously. This is a good thing as the Debian packaging system generally doesn't support package downgrades.
This is the situation I was in:
- I had a working Xen setup with linux-image-2.6.16-1-xen-686.
- A new version of this package linux-image-2.6.16-2-xen-686 came out.
- I upgraded to the new version and had an unbootable system.
- Booting with Knoppix let me attempt recovery
- But the -1- package wasn't on the local machine.
- And it had also been
removedreplaced on the Debian mirrors
Had I merely wanted to recover the machine I would have installed a non-Xen kernel upon the system and been back up and running. But without Xen I couldn't complete several important jobs, so I really wanted to revert back to the non-broken version of the package.
This is the point where I remembered that the Debian snapshot server existed.
Searching the archive for the old revision gave me some lines to add to my sources.list file:
deb http://snapshot.debian.net/archive pool linux-2.6 deb-src http://snapshot.debian.net/archive pool linux-2.6
Using these lines recovery became simple:
apt-get update apt-get install linux-headers-2.6.16-1-xen \ linux-image-2.6.16-1-xen-686 \ linux-modules-2.6.16-1-xen-686
Once those downloads completed I could rebuild my deleted initrd image and reboot into a working Xen system.
Of course if you deliberately run "older" packages you might discover that these get upgraded the next time you run "apt-get upgrade". Thankfully this is simple to avoid by holding your packages.