Rescuing systems using the Debian snapshot server
Posted by Steve on Mon 28 Aug 2006 at 08:46
One of the unofficial Debian project resources which doesn't get the attention it deserves is the Debian Snapshot site. The site contains a mirror of old Debian packages, which can be very useful for system recovery.
In most normal cases you won't ever need to use it, unless you're wanting to compare two different package versions to see changes, or do other non-standard things. However when you do need to use it you'll learn what a big lifesaver it is!
The biggest use for the site, for me, has been for recovering from broken package updates. Whilst these are rare in the Debian Stable and Testing releases they can be an issue when running Debian unstable.
As a cromulent example the Debian Xen packages available in unstable have recently become broken - they panic on boot. Reverting the package was difficult since the working copy had vanished from the Debian package repository, but the snapshot service allowed me to recover easily.
In my case this was the only option since the local package cache (in /var/cache/apt/archives) had been cleaned already - and the previous package was unavailable on the Debian mirror.
Thankfully kernel packages are nicely built such that you can have multiple versions installed simultaneously. This is a good thing as the Debian packaging system generally doesn't support package downgrades.
This is the situation I was in:
- I had a working Xen setup with linux-image-2.6.16-1-xen-686.
- A new version of this package linux-image-2.6.16-2-xen-686 came out.
- I upgraded to the new version and had an unbootable system.
- Booting with Knoppix let me attempt recovery
- But the -1- package wasn't on the local machine.
- And it had also been
removedreplaced on the Debian mirrors
Had I merely wanted to recover the machine I would have installed a non-Xen kernel upon the system and been back up and running. But without Xen I couldn't complete several important jobs, so I really wanted to revert back to the non-broken version of the package.
This is the point where I remembered that the Debian snapshot server existed.
Searching the archive for the old revision gave me some lines to add to my sources.list file:
deb http://snapshot.debian.net/archive pool linux-2.6 deb-src http://snapshot.debian.net/archive pool linux-2.6
Using these lines recovery became simple:
apt-get update
apt-get install linux-headers-2.6.16-1-xen \
linux-image-2.6.16-1-xen-686 \
linux-modules-2.6.16-1-xen-686
Once those downloads completed I could rebuild my deleted initrd image and reboot into a working Xen system.
Of course if you deliberately run "older" packages you might discover that these get upgraded the next time you run "apt-get upgrade". Thankfully this is simple to avoid by holding your packages.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Probably not, but sometimes I just can't resist.
[ Parent | Reply to this comment ]
What about 'embiggen', eh?
Have you both studied at George Bush's?
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Converting M$ addicts one CD at a time.
[ Parent | Reply to this comment ]
Thanks for this article. I never realised the snapshot server existed. It would have come in very useful in past situations. Oh well ...
I don't know if it was done on purpose, but you didn't mention anything about the /var/cache/apt/archives/ directory. This directory has solved most of my upgrade woes in the past. The only time it hasn't, is when I've done a apt-get clean or autoclean, which deletes downloaded deb files from the archives directory.
[ Parent | Reply to this comment ]
So, silly me, in order to be able to go back, I never use
apt-get autoclean
again after I needed to go back once.
thanks for the info.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
I did mention that directory - although only briefly and to say that mine had been cleaned.
(I have a cronjob that runs every night which runs :
apt-get clean apt-get update apt-get --download-only --yes upgrade
So the next morning I can login and run "apt-get upgrade" without waiting for the (potentially) large download. The downside to this is that I lose if there is a package I need in the cache ..
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
-- Jean-Marc Liotier.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
^^^^ this is a good way of ensuring a safty net. :-)
[ Parent | Reply to this comment ]