New User? Register here - Existing Users: Username: Password: [Advanced Login]

 

 

Current Poll

Your preferred Interactive shell?









( 1216 votes ~ 14 comments )

 

Offline Package Management for APT

Posted by riteshsarraf on Sat 24 Oct 2009 at 14:43

Tags: ,

This article is about Offline Package Management in Debian. Debian is a pretty well known project. One of the things that makes Debian very popular is APT (a.k.a Advanced Packaging Tool) which allows remote package downloads, upgrades and dependency resolution. Unfortunately it does require a network connection - unless you use apt-offline.

In Debian, when you need to install a package, you usually would fire up the apt-get command and the software would just install without any hand holding.

While APT is really very cool one of the main reasons for its success is the Debian Policy. The Debian Policy is like the brain of the project that controls the entire project ensuring that all the bits and pieces fit well together upto the Debian Standards. APT is just a result of the fantastic Debian Policy work.

In Debian, every package is very well self-contained and is tightly related to each other using APT. APT does a very good job of integrating and resolving dependencies for Package Management and takes off all the Dependency Hell problems from the user.

This is where the problem starts - for a machine which has network access it works very well because APT generates the list of packages and their dependencies and is able to download and install them successfully.

But when it comes to downloading a package individually on a different machine, along wih resolving any dependencies this can be a big problem.

Consider this real world example: I have a Debian box at home. At home, I have no (or very slow/expensive dial-up) internet connection. At work, I (or my friend) do have a very fast connection but (as part of IT policy) am required to use Windows.

I would still like to be able to painlessly update/upgrade my Debian box at home, with all the power and flexibility of APT.

This is where apt-offline is useful. apt-offline is an offline APT Package Manager.

Using apt-offline:

  • You generate a signature on your Debian box at home and carry the signature file on a removable medium (Probably a USB Stick).(e.g. "apt-offline set /tmp/apt-offline.txt")
  • Now you take the USB Stick (with the apt-offline.txt signature file) to the office machine which could be running any linux version, or as I mentioned above, even Windows.
  • There, you could run apt-offline giving it the signature file. (e.g. "apt-offline get C:\apt-offline.txt")
  • apt-offline would generate you an archive file or a folder with all the data. That data can be copied on a removable media. The removable media can be attached back to the disconnected Debian box at home and installed. (e.g. "apt-offline install /tmp/apt-offline.zip")

Let's start with a 3 step example

Step 1

Generate a signature file on the Disconnected Debian box at home

apt-offline set /tmp/apt-offline.sig

The above command will generate all information required from apt about updating its database.

By default, with no additional arguments passed, apt-offline will extract information about APT Package Database Update i.e. the --update option as well as the list of Packages to be upgraded i.e. the --upgrade option.

These options can also be individually passed if you want only one of those.

Step 2

Download data based on the signature file generated earlier

apt-offline get C:\apt-offline.sig --threads 5

The above command will download data as mentioned in the signature file. To speed up downloads (that can be from multiple apt repositories), in this example we spawn 5 download threads.

Note: It would be good to also download the bug reports for the packages that you are downloading. So that example now becomes:

apt-offline get C:\apt-offline.sig --bug-reports --threads 5

There are many more options that you can pass to apt-offline, like the --bundle option which would generate for you, an archive file with all the data.

Once completed, you could just copy the data (an archive file, if you used the --bundle option) back to the removable medium and copy it back onto your offline host.

Step 3

Once you're back upon the home Debian machine, you feed the data from the removable medium to apt-offline:

apt-offline install /media/USB/apt-offline.zip

This will update the APT database on your disconnected machine seamlessly.

If there were packages that needed to be upgraded, now they would all be available (with dependencies) in the APT database. So if you do an apt-get upgrade now, APT won't prompt you mentioning even a single bye download. APT would find that all required packages are already present in the APT cache.

If you had used the --bug-reports switch that I mentioned earlier, during install apt-offline would prompt you with the list of bug reports related to the packages on your machine that need be upgraded/installed - not just the list but the full bug report will be available for you to look at and evaluate the severity involved.

As you can see from the article above, apt-offline helps you achieve the power of APT, in just 3 steps. apt-offline is part of Debian and is developed at Alioth.

Share/Save/Bookmark


Posted by Anonymous (80.104.xx.xx) on Sat 24 Oct 2009 at 17:53
Does it work with aptitude, too?

thanks

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Sat 24 Oct 2009 at 20:07
[ Send Message ]
At the moment, apt-offline internally uses apt-get only.
My plans are to directly use python-apt if available. Support for aptitude is also planned, I just need to find out a way to generate the list of URIs in aptitude.

[ Parent | Reply to this comment ]

Posted by Anonymous (80.104.xx.xx) on Sat 24 Oct 2009 at 17:55
Does it work with aptitude, too?

thanks

[ Parent | Reply to this comment ]

Posted by ajt (195.112.xx.xx) on Mon 26 Oct 2009 at 22:31
[ Send Message | View Weblogs ]

That's exactly what I wanted a few years ago. As it happens the PC I need it for now has fast network and I've migrated to using aptitude for all my upgrades now.

I can still see a need for this tool though, there is nothing worse than installing a box knowing that you have a nasty wait for the upgrades to come down a slow pipe - and a local cache of updated packages would always be useful.

--
"It's Not Magic, It's Work"
Adam

[ Parent | Reply to this comment ]

Posted by Anonymous (212.179.xx.xx) on Tue 3 Nov 2009 at 14:55
Great guide but here's an error (I think):

The signature file is called apt-offline.sig on the first step and then on the next steps it's called apt-offline.txt

Thanks

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Tue 3 Nov 2009 at 19:10
[ Send Message ]
Thanks. I have fixed the same.

[ Parent | Reply to this comment ]

Posted by Anonymous (93.158.xx.xx) on Tue 17 Nov 2009 at 22:19
How are you running apt-offline in windows exactly ? This is really a tool that might interest me, so I am curious of the windows part of the process. How have you been able to install it ?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.166.xx.xx) on Wed 18 Nov 2009 at 05:05
[ Send Message ]
There is only 1 method (i.e. Step 2) that can be run under Windows or any other platform. As you see in the example for Step 2, the path to the signature file itself is a Windows path.

Step 2 is the get operation, whose sole purpose is to only fetch the required data (debs, bug reports, apt package database updates) from the internet.

You will need python installed on the windows box. Then just download the apt-offline zip archive from the alioth project page, unzip it and execute it.

[ Parent | Reply to this comment ]

Posted by huats (93.158.xx.xx) on Wed 18 Nov 2009 at 20:07
[ Send Message ]
ok Thanks the info... So it suits my needs !

[ Parent | Reply to this comment ]

Posted by goumba (72.229.xx.xx) on Tue 22 Dec 2009 at 21:33
[ Send Message ]
Its not necessary for Python to be _installed_...

At work (non-tech related business), users can not install applications on the PC and we're given a minimal amount of software of which Python is not included.

You can put Python on the same stick you plan to download to, and just add X:\<Python Directory>\bin to PATH and continue as normal.

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Wed 23 Dec 2009 at 03:27
[ Send Message ]
Yes, that's a good point. Thanks for pointing out.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.70.xx.xx) on Mon 15 Feb 2010 at 10:57
Can some help me to build and install this on windows xp?

View this image to see what message i take if i try to build it.
http://f.imagehost.org/0646/image.jpg

Sorry for my english :/.

[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.79.xx.xx) on Mon 15 Feb 2010 at 18:14
[ Send Message ]
Can you cd into the "apt-offline" folder and then try to build it ?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.79.xx.xx) on Tue 16 Feb 2010 at 09:48
[ Send Message ]
Actually, instead of a build, you might just want to cd into the apt-offline folder and run your commands, like:

c:\apt-offline> python apt-offline set C:\set.uris

There were some minor issues with the setup.py file. They have been fixed and should be part of the next (0.9.7) release.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.71.xx.xx) on Tue 16 Feb 2010 at 13:41
I have build it in windows and on Debian but every time i type apt-offline in my Debian i take some errors :/

Please see this image http://i.imagehost.org/view/0940/Screenshot

[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.79.xx.xx) on Tue 16 Feb 2010 at 17:28
[ Send Message ]
I am not sure why you are seeing this problem. I cannot see a reason why it cannot find out the module.

Are you installing it through the deb package that is available in Debian at:
http://packages.debian.org/search?keywords=apt-offline

From the screenshot it looks like, it is installed under /usr/lib/python2.5/site-packages/ but you seem to also be in ~/Desktop/apt-offline/ folder, which gives the impression that it might be an external installation.

Please file a bug report at http://apt-offline.alioth.debian.org in case you still see the problem.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.71.xx.xx) on Thu 18 Feb 2010 at 13:25
I reinstalled debian and now operates normally.
I do not know why there was a problem before, maybe I had something wrong

[ Parent | Reply to this comment ]

Posted by Anonymous (149.32.xx.xx) on Tue 23 Mar 2010 at 01:21
How do you use this to install new packages? On my disconnected Linux I tried:

apt-offline set --install-packages rcconf -- /tmp/apt-offline.sig

and I get the following error: Couldn't find package rcconf

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Tue 23 Mar 2010 at 05:10
[ Send Message ]
Looks like your apt database is empty on the disconnected Linux box. You first need to update the apt database.

For that, run:

apt-offline set /tmp/apt-offline.sig --update

It will generate the .sig file. Then run the get operation on the network machine. Then run the install operation on the disconnected Linux box with the data file.

Once the apt database is available, apt-offline will be able to generate the dependency for rcconf and its dependency packages.

[ Parent | Reply to this comment ]

Posted by Anonymous (149.32.xx.xx) on Wed 24 Mar 2010 at 00:34
Thanks for the fast response, that works. This is an amazing program. One note though, on windows "python setup.py install" had a syntax error with Python 3.1.1 but it worked with Python 2.5.4.

[ Parent | Reply to this comment ]

Posted by Anonymous (109.193.xx.xx) on Sat 27 Mar 2010 at 13:51
Hi!
Actually I have several problems with apt-offline.

When I start apt-offline get apt-offline.sig
the following errors occured:

debian:/usr# apt-offline get apt-offline.sig
/usr/lib/python2.5/site-packages/apt/__init__.py:18: FutureWarning: apt API not stable yet
warnings.warn("apt API not stable yet", FutureWarning)

Fetching APT Data

Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
self.run()
File "/usr/lib/python2.5/threading.py", line 446, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/pymodules/python2.5/apt_offline_core/AptOfflineLib .py", line 578, in run
self.responseQueue.put( self.WorkerFunction( item, thread_name ) )
File "/usr/lib/pymodules/python2.5/apt_offline_core/AptOfflineCor eLib.py", line 540, in DataFetcher
(url, file, download_size, checksum) = stripper(item)
File "/usr/lib/pymodules/python2.5/apt_offline_core/AptOfflineCor eLib.py", line 304, in stripper
size = int(string.rstrip(string.lstrip(''.join(item[2]), chars = "'"), chars="'"))
ValueError: invalid literal for int() with base 10: '5.0.4'


Downloaded data to /tmp/apt-offline-downloads-3467


Can anyone help me?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.166.xx.xx) on Sat 27 Mar 2010 at 16:14
[ Send Message ]
Can you confirm the version of apt-offline that you are using ?

And the content of the apt-offline.sig file, can you post it ?

[ Parent | Reply to this comment ]

Posted by Anonymous (109.193.xx.xx) on Sat 27 Mar 2010 at 16:41
Hi!
I installed apt-offline_0.9.7_all.deb with python-argparse_1.1-1_all.deb and python-support_1.0.7_all.deb

Here the content of the .sig-File:

'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/m ain/binary-i386/Packages.bz2'; volatile.debian.org_debian-volatile_dists_lenny_volatile_main_bin ary-i386_Packages 0 :
'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/c ontrib/binary-i386/Packages.bz2'; volatile.debian.org_debian-volatile_dists_lenny_volatile_contrib_ binary-i386_Packages 0 :
'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/m ain/source/Sources.bz2'; volatile.debian.org_debian-volatile_dists_lenny_volatile_main_sou rce_Sources 0 :
'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/c ontrib/source/Sources.bz2'; volatile.debian.org_debian-volatile_dists_lenny_volatile_contrib_ source_Sources 0 :
'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/R elease'; volatile.debian.org_debian-volatile_dists_lenny_volatile_Release 0
'http://volatile.debian.org/debian-volatile/dists/lenny/volatile/R elease.gpg'; volatile.debian.org_debian-volatile_dists_lenny_volatile_Release. gpg 0
'cdrom:[Debian GNU/Linux 5.0.4 _Lenny_ - Official i386 DVD Binary-1 20100131-19:16]/dists/lenny/contrib/binary-i386/Packages.bz2' Debian%20GNU_Linux%205.0.4%20%5fLenny%5f%20-%20Official%20i386%20 DVD%20Binary-1%2020100131-19:16_dists_lenny_contrib_binary-i386_P ackages 0 :
@

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.166.xx.xx) on Sat 27 Mar 2010 at 18:16
[ Send Message ]
Thank you. You have uncovered a bug. And it has been fixed in my git repo.

Problem is, you have the cdrom repository enabled in your apt database. For now, if you want to use apt-offline, you will have to disable the cdrom repository in your apt sources.list and rerun the set command.

In the next release (0.9.8), apt-offline will check and skip the cdrom repository items. Thanks again for uncovering this.

[ Parent | Reply to this comment ]

Posted by Anonymous (220.232.xx.xx) on Thu 30 Sep 2010 at 07:56
I need to run "apt-get install XXX" on Debian at home. XXX is not installed by default. And I have no internet connection at home. Can apt-offline help me?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (124.41.xx.xx) on Thu 30 Sep 2010 at 08:12
[ Send Message ]
Yes, That is exactly what apt-offline is there for. Just follow the howto and you should be able to get your work done.

[ Parent | Reply to this comment ]

Posted by Anonymous (90.130.xx.xx) on Sun 10 Oct 2010 at 20:51
This is what I get when running with Phyton 3.1.2 on Windows XP:


C:\Python31\Lib\site-packages\apt-offline>python apt-offl ine get .\tmp\apt-offline.sig --threads 5
Traceback (most recent call last):
  File "apt-offline", line 25, in&n bsp;<module>
    from apt_offline_core.AptOfflineCore Lib import main
  File "C:\Python31\Lib\site-packages\apt-offline\ apt_offline_core\AptOfflineCor
eLib.py", line 242
    except socket.timeout, timeout:
            ;          &nbs p;   ^
SyntaxError: invalid syntax


What to do?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.3.xx.xx) on Tue 12 Oct 2010 at 11:11
[ Send Message ]
I haven't yet done the testing with Python 3.x. Can you run it with Python 2.x ? If you still see the problem, please report it.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.71.xx.xx) on Mon 18 Oct 2010 at 16:08
Hi, I can not update my debian with apt-offlien anymore :/. When I try to update I get this error message "ERROR: package_name Checksum mismatch".
Is this a problem of apt-offline or it is other problem?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Tue 19 Oct 2010 at 18:57
[ Send Message ]
Thanks. I recently fixed this bug. I just uploaded apt-offline, with this fix, to the archive. If you installed apt-offline through a tarball, please let me know. I will provide you a link to the new tarball with the fix.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.71.xx.xx) on Fri 22 Oct 2010 at 13:24
Please post the link when you can.

[ Parent | Reply to this comment ]

Posted by Anonymous (94.71.xx.xx) on Sat 23 Oct 2010 at 13:58
Thanks! It works now.

[ Parent | Reply to this comment ]

Posted by SpiritualMadMan (138.162.xx.xx) on Fri 12 Aug 2011 at 19:56
[ Send Message ]
Unable to use apt-offline at work. There was a oneline bug that we cleared up. But, the IT protocols won't let apt-offline download any files.

I can manually get the files one by one by using the url's in IE.

But, haven't a clue what to do with them once I get them.

Any pointers on how to assemble the files so that apt-offline can use them on the Linux side would be most helpful.

Thanks!

BTW: Supporting Linux in a non-net-connected system. So, we really need a way to get the updates on a locked down Windows Machine.

[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.3.xx.xx) on Fri 19 Aug 2011 at 11:27
[ Send Message ]
Can you check and confirm if IE has proxy settings defined ?

And if so, what kind of proxy it is?

[ Parent | Reply to this comment ]

Posted by Anonymous (125.108.xx.xx) on Fri 19 Aug 2011 at 09:53
//I need to run "apt-get install XXX" on Debian at home. XXX is not installed by default. And I have no internet connection at home. Can apt-offline help?

I cann't get the apt-offline work for me,I want install emacs,samba.But when I follow the steps,I cann,t install them too!
this is the way I did:
on ubuntu(no internet):
apt-offline set /tmp/apt-off-emacs.sig --release emacs
on windows(fast internet):
apt-offline get C:\\apt-off-emacs.sig --threads 2 (I got a folder)
back ubuntu:
apt-offline install /home/...(the folder)
and after a while when the apt-offline install command finished,nothing have been changed! After that I tried apt-get install emacs,nothing happened too!
so can you give me an example of how I get specific soft like emacs!
because I cann't find the example!!
thank you!





[ Parent | Reply to this comment ]

Posted by riteshsarraf (202.3.xx.xx) on Fri 19 Aug 2011 at 11:25
[ Send Message ]
Please follow the howto correctly. It will work.
from what you have mentioned, --release option requires a valid release, something like testing, unstable or stable.
So your correct command would be: "apt-offline set /tmp/apt-off-emacs.sig --release unstable --install-packages emacs samba"
More details available with: apt-offline set -h

[ Parent | Reply to this comment ]

Posted by Anonymous (125.108.xx.xx) on Sat 20 Aug 2011 at 00:35
Please post the "howto" link,where the "howto"?in the apt-offline package?

[ Parent | Reply to this comment ]

Posted by riteshsarraf (122.167.xx.xx) on Sat 20 Aug 2011 at 08:13
[ Send Message ]
There's no other help apart from this document, the manpage and the "-h" help opiton.

[ Parent | Reply to this comment ]

 

 

Flattr