Posted by Steve on Sat 5 Nov 2005 at 16:04
We've previously covered setting up your own repository for the Debian's apt-get system, but we didn't cover managing automatic uploads. Thankfully this is a simple task with the reprepro, and dupload tools and a small amount of scripting.
The reprepro package is tool for creating an APT repository with a pool structure, the same type of structure the official Debian mirrors use.
The repository may:
Removing PackagesInstalling the package is straightforward if you're using unstable, or etch, simply install it as you would install any other package:
apt-get install repreproUnfortunately the package contained within Debian's stable release, sarge, is a little outdated. For that reason I'd recommend that you install a back ported version. Thankfully this is readily available either from the projects homepage or elsewhere:
Once you've installed the software then we're ready to begin creating a repository. A new repository can be created in just a couple of steps:
- Decide where you wish the archive to be located.
- Create the configuration file.
- Import your first package
Almost certainly you will wish to serve your repository via a webserver, so the location should be beneath your webservers root directory at a simple path.
I use http://steve.org.uk/apt as my repository root location so my archive will be located beneath /home/www/www.steve.org.uk/htdocs/apt - your location will obviously differ.
First of all create the directory, a subdirectory conf/ to contain the configuration file, and a directory incoming/ to which we'll later setup automatic upload processing:
mkdir -p /home/www/www.steve.org.uk/htdocs/apt mkdir -p /home/www/www.steve.org.uk/htdocs/apt/conf mkdir -p /home/www/www.steve.org.uk/htdocs/apt/incomingNow that we have a directory to contain our repository we can look at creating the configuration file. The configuration file will specify which releases the repository will contain (sid, stable, etc) as well as the architectures. A sample configuration file will look like this:
Origin: Your Name Label: Your own label Suite: stable Codename: sarge Version: 3.1 Architectures: i386 all source Components: main non-free contrib Description: Your descriptionHere we've defined a repository which only contains packages for stable/sarge, which contains packages targeted to x86, or all. If you wish to contain both stable and unstable packages your configuration file will look like this:
Origin: Your Name Label: Your own label Suite: stable Codename: sarge Version: 3.1 Architectures: i386 all source Components: main non-free contrib Description: Your description Origin: Your Name Label: Your own label Suite: unstable Codename: sid Architectures: i386 all source Components: main non-free contrib Description: Your descriptionSave your configuration to the file conf/distributions and you should now be ready to import a package.
You can either import a .deb file into the repository, or a .changes file which is produced by building a package from source.
From the main directory run:
reprepro -Vb . include sarge name_of_fileFor example:
skx@lappy:~/apt$ reprepro -Vb . include sarge \ /home/skx/debian/sarge/reprepro/reprepro_0.6-1sarge0_i386.changes Created directory "./db" Created directory "./pool" Created directory "./pool/main" Created directory "./pool/main/r" Created directory "./pool/main/r/reprepro" db: 'reprepro' added to 'sarge|main|i386'. db: 'reprepro' added to 'sarge|main|source'. Created directory "./dists" Created directory "./dists/sarge" Created directory "./dists/sarge/main" Created directory "./dists/sarge/main/binary-i386" writing to './dists/sarge/main/binary-i386/Packages.new'... writing to './dists/sarge/main/binary-i386/Packages.gz.new'... Created directory "./dists/sarge/main/binary-all" writing to './dists/sarge/main/binary-all/Packages.new'... writing to './dists/sarge/main/binary-all/Packages.gz.new'... Created directory "./dists/sarge/main/source" writing to './dists/sarge/main/source/Sources.gz.new'... Created directory "./dists/sarge/non-free" Created directory "./dists/sarge/non-free/binary-i386" writing to './dists/sarge/non-free/binary-i386/Packages.new'... writing to './dists/sarge/non-free/binary-i386/Packages.gz.new'... Created directory "./dists/sarge/non-free/binary-all" writing to './dists/sarge/non-free/binary-all/Packages.new'... writing to './dists/sarge/non-free/binary-all/Packages.gz.new'... Created directory "./dists/sarge/non-free/source" writing to './dists/sarge/non-free/source/Sources.gz.new'... Created directory "./dists/sarge/contrib" Created directory "./dists/sarge/contrib/binary-i386" writing to './dists/sarge/contrib/binary-i386/Packages.new'... writing to './dists/sarge/contrib/binary-i386/Packages.gz.new'... Created directory "./dists/sarge/contrib/binary-all" writing to './dists/sarge/contrib/binary-all/Packages.new'... writing to './dists/sarge/contrib/binary-all/Packages.gz.new'... Created directory "./dists/sarge/contrib/source" writing to './dists/sarge/contrib/source/Sources.gz.new'...This has imported the package described in the file reprepro_0.6-1sarge0_i386.changes to the archive, creating the appropriate directories as required.
To have a less verbose output simply omit the -V flag, thusly:
skx@lappy:~/apt$ reprepro -b . include sarge \ /home/skx/debian/sarge/reprepro/reprepro_0.6-1sarge0_i386.changes skx@lappy:~/apt$
Using Your RepositoryIf you wish to remove a package from your repository you can do so with the remove command:
skx@lappy:~/apt$ reprepro -b . remove sarge reprepro Deleting files no longer referenced... deleting and forgetting pool/main/r/reprepro/reprepro_0.6-1sarge0_i386.deb deleting and forgetting pool/main/r/reprepro/reprepro_0.6-1sarge0.dsc deleting and forgetting pool/main/r/reprepro/reprepro_0.6.orig.tar.gz deleting and forgetting pool/main/r/reprepro/reprepro_0.6-1sarge0.diff.gzNote: you don't need to remove a package if you simply wish to replace an existing package with a newer version - this will be handled for you. When you import a newer version of a package contained in the archive already the older version will be removed.
Configuring Package Uploads With duploadOnce your packages has been added to the archive they may be downloaded via apt-get, or aptitude, with the appropriate lines in your /etc/apt/sources.list file:
deb http://example.com/apt sarge main contrib non-free deb-src http://example.com/apt sarge main contrib non-freeIf you're using two distributions simply repeat for each:
deb http://example.com/apt sarge main contrib non-free deb-src http://example.com/apt sarge main contrib non-free deb http://example.com/apt sid main contrib non-free deb-src http://example.com/apt sid main contrib non-freeYou should find that packages can be downloaded and installed as expected.
Importing Packages from an Incoming queue with repreprodupload is a tool which is designed to allow you to upload packages to different repositories.
Once installed (via "apt-get install dupload") it may be configured either via the file /etc/dupload.conf or ~/.dupload.conf.
To configure uploads for your remote host then simply add a section such as this:
$cfg{'example'} = { fqdn => "example.com", login => "steve", method => "scpb", incoming => "/incoming/", # The dinstall on ftp-master sends emails itself dinstall_runs => 1, };This configuration :
- Names the upload target 'example'.
- Specifies that file uploads should be conducted using scp.
- With the login name steve.
- That packages should be uploaded into the directory /incoming.
You will certainly need to change the hostname, login name, and incoming directory. You might also wish to specify an alternative means of uploading, such as anonymous FTP. For more details of the available options please see "man dupload.conf".
Once you've configured the upload settings you should be able to upload a Debian package by executing:
dupload --to example \ /home/skx/debian/sarge/reprepro/reprepro_0.6-1sarge0_i386.changes(Note that you should upload the .changes file, not the .deb file)
Once this command completes you'll find that you've successfully transferred the package to your incoming directory, beneath your apt root.
The next step is to allow reprepro to automatically add that package to the repository.
As a result of any dupload commands we'll expect to process the incoming files from the incoming/ directory. We've already seen that reprepro can handle the importing of packages via the .changes file.
So we simply need to write a small shell script which will locate each .changes file, import the package, then clean the directory afterwards.
(We must remove, or move, the files from the incoming directory to make sure we don't repeatedly try to re-add the same package.)
This is a sample script which does the job. It is based upon the one I use myself and should be easily usable elsewhere - simply change the incoming directory at the top of the script:
#!/bin/sh INCOMING=/home/www/www.steve.org.uk/htdocs/apt/incoming # # Make sure we're in the apt/ directory # cd $INCOMING cd .. # # See if we found any new packages # found=0 for i in $INCOMING/*.changes; do if [ -e $i ]; then found=`expr $found + 1` fi done # # If we found none then exit # if [ "$found" -lt 1 ]; then exit fi # # Now import each new package that we *did* find # for i in $INCOMING/*.changes; do # Import package to 'sarge' distribution. reprepro -Vb . include sarge $i # Delete the referenced files sed '1,/Files:/d' $i | sed '/BEGIN PGP SIGNATURE/,$d' \ | while read MD SIZE SECTION PRIORITY NAME; do if [ -z "$NAME" ]; then continue fi # # Delete the referenced file # if [ -f "$INCOMING/$NAME" ]; then rm "$INCOMING/$NAME" || exit 1 fi done # Finally delete the .changes file itself. rm $i doneTo use this simply setup a cronjob running every few minutes to process any new files in the incoming/ directory - something like this:
*/5 * * * * /usr/local/bin/import-new-packages.sh
This article can be found online at the Debian Administration website at the following bookmarkable URL:
This article is copyright 2005 Steve - please ask for permission to republish or translate.