Modifying APT: logging and proxy server usage

Posted by hruske on Thu 7 Jul 2005 at 01:40

Since dpkg only acquired logging facilities with version 1.13, which didn't make it to Sarge, I'm going to present a way to get package installation logged.

Apt has a nice configuration system, which in Debian is split among more files located in /etc/apt/apt.conf.d/. This can be very helpful when implementing simple logging.

If we put a file in the directory /etc/apt/apt.conf.d/ with following content:

DPkg::Pre-Install-Pkgs {"/usr/local/bin/log-apt-get ";};

Apt will then execute command /usr/local/bin/log-apt-get before installing every package. Apt also has similar configuration options such as DPkg::Pre-Invoke and DPkg::Post-Invoke, but these are not really suitable for logging, as only Pre-Install-Pkgs passes package name (path) on to command (to stdin).

Now we need to create the log-apt-get command to record the information:

#!/bin/bash
while read paket;
  do echo $(date +'%b %e %R:%S') ${paket#/var/cache/apt/archives/} >> /var/log/apt-get.log;
done

Package installations are now logged in the following following format:

Jul 7 03:44:20 less_382-2_amd64.deb
Jul 7 03:45:06 gzip_1.3.5-11_amd64.deb

NOTE: There are a few drawbacks. Since this is an Apt hack, it will only log installs made with Apt. If a package is installed directly with dpkg, it will not be logged. Also this is a much less verbose log compared to log made by dpkg version 1.13.

This log will also not contain entries for package removal.

(If you're using aptitude you can find it's logs located in the file /var/log/aptitude.

Downloading Debian Packages through a Proxy

Anothing example of using the apt configuration files is to instruct apt-get to download packages via a proxy server.

Simply place the following into a file such as /etc/apt/apt.conf.d/proxy:

Acquire::http::Proxy "http://proxy:8080";

For an FTP proxy add a stanza like this, but be sure, to change only proxy hostname and port.

Acquire::ftp
{
  Proxy "ftp://proxy:2121/";
  ProxyLogin
  {
     "USER $(SITE_USER)@$(SITE)";
     "PASS $(SITE_PASS)";
  }
}

 

 


Posted by shufla (83.30.xx.xx) on Thu 7 Jul 2005 at 20:59
Hi,

If you are able to set up apt-proxy(8) that's better. It needs more configuration, but according to apt-proxy(8) man page it has some nice featuers.


Best Regards,
Lukasz

[ Parent | Reply to this comment ]

Posted by Anonymous (150.101.xx.xx) on Fri 8 Jul 2005 at 06:11
Note that aptitude(1), which *was* released in sarge, logs any package action you ask it to perform (to /var/log/aptitude by default).

[ Parent | Reply to this comment ]

Posted by ybiC (68.96.xx.xx) on Fri 8 Jul 2005 at 14:22

Yeppers, aptitude was even available back with Woody, from official Debian repositories.

The first respsone I generally hear, on suggesting aptitude, is "It's hard to navigate and I don't like the dselect-ish menu interface stuff", when of course aptitude can be used from the commandline almost exactly like apt-get.

aptitude update
aptitude upgrade
aptitude dist-upgrade
/bin/su -c 'aptitude update && aptitude upgrade'

And, as described above, aptitude logs all of its package mangement actions.

[ Parent | Reply to this comment ]

Posted by ybiC (68.96.xx.xx) on Fri 8 Jul 2005 at 14:35
And "aptitude help" provides the following swell info, perhaps a wee bit more readable than "man aptitude".
aptitude ver.number
Usage: aptitude [-S fname] [-u|-i]
       aptitude [options]  ...
  Actions (if none is specified, aptitude will enter interactive mode):

 install      - Install/upgrade packages
 remove       - Remove packages
 purge        - Remove packages and their configuration files
 hold         - Place packages on hold
 unhold       - Cancel a hold command for a package
 markauto     - Mark packages as having been automatically installed
 unmarkauto   - Mark packages as having been manually installed
 forbid-version - Forbid aptitude from upgrading to a specific package version.
 update       - Download lists of new/upgradable packages
 upgrade      - Perform a safe upgrade
 dist-upgrade - Perform an upgrade, possibly installing and removing packages
 forget-new   - Forget what packages are "new"
 search       - Search for a package by name and/or expression
 show         - Display detailed information about a package
 clean        - Erase downloaded package files
 autoclean    - Erase old downloaded package files
 changelog    - View a package's changelog
 download     - Download the .deb file for a package

  Options:
 -h             This help text
 -s             Simulate actions, but do not actually perform them.
 -d             Only download packages, do not install or remove anything.
 -P             Always prompt for confirmation or actions
 -y             Assume that the answer to simple yes/no questions is 'yes'
 -F format      Specify a format for displaying search results; see the manual
 -O order       Specify how search results should be sorted; see the manual
 -w width       Specify the display width for formatting search results
 -f             Aggressively try to fix broken packages.
 -V             Show which versions of packages are to be installed.
 -D             Show the dependencies of automatically changed packages.
 -Z                 Show the change in installed size of each package.
 -v             Display extra information. (may be supplied multiple times)
 -t [release]   Set the release from which packages should be installed
 -o key=val     Directly set the configuration option named 'key'
 --with(out)-recommends Specify whether or not to treat recommends as
                strong dependencies
 -S fname: Read the aptitude extended status info from fname.
 -u      : Download new package lists on startup.
 -i      : Perform an install run on startup.

                  This aptitude does not have Super Cow Powers.

[ Parent | Reply to this comment ]

Posted by Anonymous (193.170.xx.xx) on Wed 13 Jul 2005 at 20:40

The proxy server configuration is even more powerfull. You can exclude several hosts, e.g. your local mirror (named "mirror" and "mirror.example.com"):

Acquire
{
  http {
    Proxy "http://proxy.example.com:8080/";
    Proxy::mirror "DIRECT";
    Proxy::mirror.example.com "DIRECT";
  }
}
There is another nice way to use passive FTP:
Acquire
{
  ftp {
    Passive "true";
  };
};

And a third Trick: The APT cache could get too small if you have many entries in your sources.list. You can increase the cache limit with:

APT::Cache-Limit 12582912;
(12582912 = 12*1024*1024) found at http://katspace.net/computers/linux_tips.php

Bye Hansi

[ Parent | Reply to this comment ]

Posted by Anonymous (41.206.xx.xx) on Wed 5 Mar 2008 at 08:15
I have a question: my proxy username is on a domain and my password contains the character @. How do I specify the username and password for the HTTP proxy?
http://domain\user:p@ssword@proxy.company:8080/ doesn't work.

[ Parent | Reply to this comment ]

Posted by Anonymous (216.136.xx.xx) on Fri 28 Mar 2008 at 15:23
I'm having this problem also, how could someone overlook that?
Escaping \@ it doesn't seem to work either.

[ Parent | Reply to this comment ]

Posted by Anonymous (200.238.xx.xx) on Mon 26 May 2008 at 20:13
same problem here,

[ Parent | Reply to this comment ]

Posted by shreyas (220.225.xx.xx) on Sat 28 Feb 2009 at 06:20
same problem here.
it's astonishing to see a problem like this persisting for almost an year, in such a big community like debian.

i wont say i am a geek or something, but with whatever limited experience and knowledge i have, i will try to fix this one. i hope this is not a fundamental design issue or something.

-
shreyas

[ Parent | Reply to this comment ]

Posted by shreyas (116.72.xx.xx) on Sun 1 Mar 2009 at 07:28
good news. i could fix this bug in apt. it was reported in debian bug tracking system with bug id #500560.

i have submitted patches for both lenny and etch versions of apt. you can get the patches at either the BTS bug #500560 page, or at this url - http://pyarabola.blogspot.com/2009/03/setting-up-debian-apt-behin d-proxy.html

[ Parent | Reply to this comment ]

Posted by Anonymous (14.139.xx.xx) on Wed 26 Oct 2011 at 05:22
Hi I am new to Linux, would you briefly explain how to use this patch.

[ Parent | Reply to this comment ]

Posted by Anonymous (85.176.xx.xx) on Tue 5 Dec 2006 at 00:18
here i found a different implementation which filtered out the package name and version: http://moto.debian.org.tw/viewtopic.php?t=3017

I combined this example here and the other one to create an apt log that can be replayed easily - see http://www.sprang.de/wiki/index.php/Proven_solutions#logging_apt- get_installs


Henning Sprang - henning underscore sprang aet gmx dot de

[ Parent | Reply to this comment ]

Posted by Anonymous (212.31.xx.xx) on Thu 29 Mar 2012 at 13:24
Thanks! It's old, but this post really helped me.

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 467 votes ~ 5 comments )