Keeping many Debian servers up to date with apt-proxy
Posted by kgfullerton on Mon 23 Jan 2006 at 14:10
Maybe, like me, you've got more than one Debian box on your network - either at home or at work and you want to keep them up to date with apt but are on a slow link or metered bandwidth? If so, apt-proxy could be the answer for you.
apt-proxy is a Python based daemon that caches all apt requests that go through it, and stores a copy of the files locally, so you only need to download a copy of the .deb via apt once.
Installation is simple - just run
apt-get install apt-proxyon the machine you want as your proxy server. After installation is complete you'll have a daemon listening on port 9999.
Packages downloaded are stored in /var/cache/apt-proxy by default, but this and many other options, including the servers to use to download from, can be changed in the config file /etc/apt-proxy/apt-proxy-v2.conf - you can change the port and the interface that apt-proxy listens on.
The only change left is to change your /etc/apt/sources.list on all Debian machines and change them to look at http://$APT_PROXY_MACHINE:9999/debian.
Now, the next apt-get update and apt-get upgrade should be fast for the rest of the machines on your network.
[ Parent | Reply to this comment ]
[ Send Message | View dopehouse's Scratchpad ]
Here is an example cut from my apt-proxy-v2.conf:
[debian]
backends =
http://ftp2.de.debian.org/debian
http://ftp.de.debian.org/debian
http://ftp.us.debian.org/debian
ftp://ftp.uk.debian.org/debian
If the ftp2.de.debian.org is down, than the next server will be used.
[ Parent | Reply to this comment ]
I wanted to try this without reading ANY of the apt-proxy documentation just to see how easily it would be to drop in for my servers
But a sources.list that just has one line
http://MYSERVER:9999/debian
didn't work. Sure, if I read the documentation, i'd probably be able to figure it out...but...that wasn't the point...
aaron
Through correctness comes ease
-Chiun
-The Destroyer series
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
HTTP Replicator is a general purpose, replicating HTTP proxy server. All downloads through the proxy are checked against a private cache, which is an exact copy of the remote file structure. If the requested file is in the cache, replicator sends it out at LAN speeds. If not in the cache, it will simultaneously download the file and stream it to multiple clients. No matter how many machines request the same file, only one copy comes down the Internet pipe. This is very useful for maintaining a cache of Debian or Gentoo packages.
so i have normal sources.list and apt.conf with Acquire::http::Proxy
for long time ago i used apt-proxy, but now i'm happy with http-replicator ... the best feature is: exact copy of the remote file structure - is this possible with approx, apt-proxy-v2 or apt-cacher? or i must for archive use apt-move? and is apt-proxy ready for Packages.diff ???
[ Parent | Reply to this comment ]
[ Send Message | View dkg's Scratchpad | View Weblogs ]
Similar to you, i have a line in /etc/apt/apt.conf.d/99local that reads something like:
Acquire::http::Proxy "http://proxy.my.local.network:3128/";
Using a generic proxy like squid seems better to me than using an apt-specific proxy because:
- it has a wider userbase than any apt-specific proxy (more debugging, better community support)
- it can be reused for other proxying tasks
- it is deliberately agnostic about the data being fetched (no changes need to be made to the proxy if the apt protocol/repository layout shifts subtly)
[ Parent | Reply to this comment ]
http-replicator is fine - i used many backports, so i made archive debian cd with used packages - so i can create cd/dvd only with my lovely packages ;-)
or i can mirror some subdirectory with all packages (e.g. xorg), then copy into http-replicator cache and i have complete archive ...
http-replicator can run in static mode:
Static mode: files are known to never change so files that are present are served from cache directly without contacting the server.
but http-replicator don't know ftp protocol ...
so if some apt proxy utility can use more failover debian mirrors, http or ftp, maintenance of mirror, no sources.list changes and exact copy of directory structure - it can be great util
i'm still using dselect, so with http-replicator i have:
DSelect::Options "-o Acquire::http::Proxy='http://my-http-replicator:8080'";;
DSelect::UpdateOptions "-o Acquire::http::Proxy='none'";
and i'm using apt-rsync too on slow link ...
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
E: Type 'http://127.0.0.1:9999/debian'; is not known on line 10 in source list /etc/apt/sources.list
Then,like me, you probably placed something silly such as:
http://127.0.0.1:9999/debian
into your /etc/apt/sources.list rather than the correctly formed:
deb http://127.0.0.1:9999/debian/ testing main
a great tutorial. thanks for sharing (c:
A Noni-Moose
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]