Cleaning up a Debian GNU/Linux system

Posted by pupeno on Thu 9 Nov 2006 at 10:04

You arrive at a Debian GNU/Linux server which has some history of neglection. Let's suppose someone else neglected it but if your new-year resolution is to stop neglecting your beloved server, this applies as well.

One form of neglection is to install, install, install and never un-install any package. The common utility to perform installation and un-installation of packages is apt-get which adds to the problem because it doesn't have automatic removal of non-needed dependences.

That means that when phpMyAdmin was installed it also installed Apache, PHP and ton of other packages. phpMyAdmin was removed when it was no longer needed but Apache, PHP and the ton of packages remain there.

Aptitude to the rescue. Aptitude is another package manager front-end like apt-get but it can keep track of automatically and non-automatically installed packages. That means that when you installed phpMyAdmin it was marked as non-auto while Apache and company was marked as auto. When you remove phpMyAdmin all the non-needed automatically installed packages like Apache would be removed.

That is nice, but since the neglecting previous administrator didn't use Aptitude, all the packages are marked as non automatically installed. The safe way so Aptitude doesn't remove anything that it is needed. So, what to do now ?

Well, the answer is: try to mark all files as automatically installed except those that you really want. To do that you can use the following (which you could write in one line if you want):

for pkg in $(aptitude search ~i | grep -v "i A" | cut -d " " -f 4) ; do
  echo "-- markauto $pkg --"
  aptitude markauto $pkg
done

A little explanation about this. This piece:

aptitude search ~i | grep -v "i A" | cut -d " " -f 4

you can run it by itself. It list all installed packages, then grep remove all those marked as automatically (we really don't care about them). The cut part extracts the name of the package.

Once you have the name of each package you print a little header to know what package we are talking about and then try to mark it as automatic with:

aptitude markauto 

in many cases marking a package as auto will not remove it, because another package depends on it, on other cases it'll remove the package and maybe some other non-needed packages. In those cases Aptitude will ask you wether you want to continue or not. That is where you'll have to do your job, analyze wether those packages are needed or not. If they are, tell Aptitude not to perform the operation and the for loop will continue with the next package.

At last, you may also consider removing the configurations of all those removed packages. Before doing this, make backups, that's very important and I meant it.

To remove the configurations you can use the following code:

for pkg in $(dpkg -l | grep ^rc | cut -d " " -f 3) ; do
  dpkg -P $pkg ;
done

At the end you'll have a system that is a bit cleaner.

This article originally appeared here.

 

 


Posted by Anonymous (62.31.xx.xx) on Thu 9 Nov 2006 at 10:48
The last snippet of code can be replaced by: aptitude purge ~c.

[ Parent | Reply to this comment ]

Posted by Anonymous (192.88.xx.xx) on Thu 9 Nov 2006 at 10:49
what about deborphan?

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Thu 9 Nov 2006 at 13:16
[ View Steve's Scratchpad | View Weblogs ]

We've certainly covered using deborphan in the past.

Personally I've never really used aptitude although I know we're supposed to now, especially for dist-upgrades. So using deborphan is my first step.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (84.155.xx.xx) on Thu 9 Nov 2006 at 11:26
The package debfoster may also be very helpfull.

Package page:
http://packages.debian.org/debfoster

Package description:
Install only wanted Debian packages

debfoster is a wrapper program for apt and dpkg. When first run, it will ask you which of the installed packages you want to keep installed.

After that, it maintains a list of packages that you want to have installed on your system. It uses this list to detect packages that have been installed only because other packages depended on them. If one of these dependencies changes, debfoster will take notice, and ask if you want to remove the old package.

This helps you to maintain a clean Debian install, without old (mainly library) packages lying around that aren't used any more.

[ Parent | Reply to this comment ]

Posted by weatheredwatcher (167.7.xx.xx) on Thu 10 Jul 2008 at 16:38
Wow! Why did I not know about this program before? Very helpful!

[ Parent | Reply to this comment ]

Posted by kevinoid (24.92.xx.xx) on Thu 9 Nov 2006 at 12:18
For a bit of variety, I believe you could also mark all non-automatically-installed packages as automatically installed and purge the unused packages by running the command
aptitude --purge-unused markauto '~i!~M'
Of course, this will probably purge nearly everything on your system... so you might want to run with --schedule-only and then (either in graphical mode or from a list of packages) mark packages that you want to keep as not automatically installed before completing the action. ;)

Also, from the aptitude manual page, a more conservative approach (for those of you who just want your system a little cleaner) is to just mark all packages in the libs section as automatically installed by running
aptitude markauto ’~slibs’
or any other, more conservative, search pattern.

[ Parent | Reply to this comment ]

Posted by pupeno (200.126.xx.xx) on Thu 9 Nov 2006 at 12:39
I should have included the following disclaimer:

"I am not a system administrator, I am only performing system administration tasks in my current job the best way I can (what I really love to do is programming). Any of my solutions are the first acceptable solutions I was able to find, any moderately experience/knowledge system administrator is likely to come up with much better and efficient solutions".

I think I'll keep all the replies bookmarked because there's a lot of nice and useful information in them. Thank you!

[ Parent | Reply to this comment ]

Posted by Anonymous (128.119.xx.xx) on Thu 9 Nov 2006 at 20:01
Probably most people would be okay with this:

aptitude markauto '~i!~M!~nbuild-essential(~E|~prequired|~sdevel|~sinterpreters|~sl ibdevel|~slibs|~soldlibs|~sperl|~spython|~sshells)'

[ Parent | Reply to this comment ]

Posted by Anonymous (59.178.xx.xx) on Fri 10 Nov 2006 at 06:03
Lovely article and reasoning behind it.

PJ

PS: English language tip: s/neglection/neglect/g

Yeah, I know, English is obviously not your first language. It's just that "neglection" is shakespearean english, so it had my brain sputtering every time I went past it ;-), but I am funny that way.

So don't worry about that. What is much more important was that the article was clear and logical.

And to be honest, I found the language glitches kind of charming, plus they help you imagine the writer is right there in front of you, explaining things.

[ Parent | Reply to this comment ]

Posted by paulgear (203.206.xx.xx) on Sat 11 Nov 2006 at 09:53
For the benefit of the original poster: "neglect" is the appropriate English form. :-)

[ Parent | Reply to this comment ]

Posted by paulgear (203.206.xx.xx) on Sat 11 Nov 2006 at 10:03
Duh - i missed the big regexp in the previous post rather badly. :-(

[ Parent | Reply to this comment ]

Posted by Anonymous (87.74.xx.xx) on Fri 10 Nov 2006 at 09:19
Neglection? Well, we don't want to neglect George Bush entirely already!

[ Parent | Reply to this comment ]

Posted by Anonymous (87.74.xx.xx) on Fri 10 Nov 2006 at 09:27
Sorry - just noticed comment above. Nice well-thought out article.

[ Parent | Reply to this comment ]

Posted by paulgear (203.206.xx.xx) on Sat 11 Nov 2006 at 10:11
Why does aptitude produce such bad results on my system? Take a look at this: http://rafb.net/paste/results/daCZa815.html

My system is a mix of sarge, sarge-backports, and a bit of testing, and apt-get thinks everything is sweet, while aptitude wants to make a bunch of totally nonsensical decisions. It wants removing what i have installed - e.g. vim, openoffice.org, dia-gnome, gedit, gpdf, planner - and install what i don't want - e.g. evolution, gnome-pilot-conduits, spamassassin, spamc. Why? Also, apt-cache policy shows the right things about my system - what's the aptitude equivalent?

Is aptitude really the "blessed" method of dist-upgrades? If so, my system is in trouble!

[ Parent | Reply to this comment ]

Posted by pupeno (200.126.xx.xx) on Sat 11 Nov 2006 at 12:40
If it wants to remove something it's because it is marked as automatically installed and nothing depends of it. If you want, you have to mark it as manually installed (unmarkauto) so aptitude won't try to uninstall it.

[ Parent | Reply to this comment ]

Posted by paulgear (203.206.xx.xx) on Sat 11 Nov 2006 at 22:16
And what about the things it wants to auto-install that i've never asked for (e.g. evolution)?

[ Parent | Reply to this comment ]

Posted by pupeno (200.126.xx.xx) on Sun 12 Nov 2006 at 00:22
If you run aptitude in interactive mode (by just issuing `aptitude`) and then try to perform the same command it'll show you the list of changes.
Clicking on each of the packages it gives you some information about why it is being uninstalled or installed. Maybe that gives you more information.

[ Parent | Reply to this comment ]

Posted by chr0nik (68.61.xx.xx) on Tue 21 Nov 2006 at 10:38
[ View Weblogs ]
is there a low or high probability of !burning up for just simply saying

ty

signal to noise already already

[ Parent | Reply to this comment ]

Posted by ppmm (160.83.xx.xx) on Wed 6 Dec 2006 at 20:07
an enhancement to apt dependency:

say
pkg p1 depends on d1, d2, d3
pkg p2 depends on d2, d3, d4
pkg p3 depends on d1, d2, d5, p1

we set aside a counter to every package that has been installed.

so, at the beginning we have:

which is nothing

after we install p1, we have:
p1: 1
d1: 1
d2: 1
d3: 1

after we install p2, we have:
p1: 1
p2: 1
d1: 1
d2: 2
d3: 2
d4: 1

after we install p3, we have:
p1: 2
p2: 1
p3: 1
d1: 2
d2: 3
d3: 2
d4: 1
d5: 1

so, if we remove p3, we know d5 can be safely removed.
or, if we remove p2, we know d4 can be removed(of course p3 will be broken and if p3 got removed due to dependency issue, so is d5 which should be removed)
or, if we remove p1, we know d1,d2,d3 will stay but their counter will be decreased by 1.

[ Parent | Reply to this comment ]

Posted by pupeno (200.126.xx.xx) on Wed 6 Dec 2006 at 22:11
Isn't that what aptitude does (probably what aptitude does is a bit more complex).

[ Parent | Reply to this comment ]

Posted by fhackenberger (84.115.xx.xx) on Sun 8 Mar 2009 at 14:16
I would like to propose an alternative approach, which caters for the fact that the original scripts needs constant attention and may run for a long time.
for pkg in $(aptitude search ~i | grep -v "i A" | cut -d " " -f 4) ; do
	if ! aptitude markauto --simulate --assume-yes $pkg 2>&1 | grep 'will be REMOVED' > /dev/null; then
		sudo aptitude markauto $pkg;
	else
		echo $pkg >> toremove.txt;
	fi
done
The script above will mark all packages as automatically installed if the action will not remove any packages. Meanwhile the script will create a list of packages which would cause the removal of packages if they were marked auto-installed. You can then remove all packages from the file toremove.txt which you would like to keep and run:
for pkg in `cat toremove.txt`; do
	sudo aptitude markauto $pkg;
done
That will still ask you for confirmation before actually removing any packages. If you are VERY sure about the toremove.txt list, you can add ''--assume-yes'' to remove them automatically.

[ Parent | Reply to this comment ]

Posted by Anonymous (153.100.xx.xx) on Wed 4 Nov 2009 at 09:49
aptitude markauto ~i should to the same trick!

[ Parent | Reply to this comment ]

Posted by Anonymous (85.127.xx.xx) on Thu 5 Nov 2009 at 21:33
i found this article and liked the idea, but for my use the mentioned methods would mark far to many packages as automatically installed

i wouldn't want to uninstall something important, just because i remove the last package that depends on it

but the following command worked great for me:
aptitude search '!~pstandard!~pimportant!~prequired!~t^desktop$!~tkde-desktop !~M~i'

it shows a list of manually installed packages, not including packages from the standard installation (priority standard, important or required) and also not including the packages from the desktop and kde-desktop task (gnome-desktop could be used as well)

in my case the list was so short, i could manually decide which packages to mark as auto

[ Parent | Reply to this comment ]

Posted by Anonymous (121.6.xx.xx) on Mon 8 Nov 2010 at 14:38
The original script will miss out packages with long names such as "openoffice.org-hyphenation-en-us". A better way to get the complete list of manually-installed packages using only aptitude is:

aptitude search "?installed ?not(?automatic)"  ;--display-format "%p"

[ Parent | Reply to this comment ]

Posted by Anonymous (220.255.xx.xx) on Wed 10 Nov 2010 at 11:14
should be aptitude search "?installed ?not(?automatic)"  ;--display-format "%p"

[ Parent | Reply to this comment ]

Posted by Anonymous (220.255.xx.xx) on Wed 10 Nov 2010 at 11:18
just remove the semicolon before "--display-format"

aptitude search "?installed ?not(?automatic)" --display-format "%p"

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 475 votes ~ 5 comments )