How packages are added to the Debian distribution

Posted by Steve on Sun 10 Jul 2005 at 19:03

Debian is well known for the software which is contained in its repositories. There are thousands of packages available already in the official archives, (and more packages held in various unofficial archives). But how does new software get added to the distribution?

Requirements

In order for a package to be "in" the Debian distribution it needs to exist upon within the official Debian package repositories. For that to happen four things need to occur:

Debian Policies

Before you begin creating the package you should first file an "ITP" bug in the Debian Bug Tracking System (BTS). This means that you Intend To Package the given software. Doing this reduces the risk of somebody else duplicating your effort by simultaneously working upon the same piece of software.

The list of packages for which ITP bugs have been filed should be viewed and searched before you start work on your package, to make sure you don't duplicate effort of somebody else needlessly.

Creating packages for your own personal use is a fairly simple process, especially if you create a Debian package with checkinstall or use some other existing tool to help such as the dh-make-perl tool for packaging Perl CPAN modules.

To have a package in the Official Debian repositories the requirements are set a bit higher than just a rough package; Your package must follow the Debian standards, and conform to the Debian policy manual.

These policy guidelines help ensure that the software which is included is of the highest possible quality.

The policy manual describes how software is named, which section of the archive it should be placed into and how different aspects of its installation should be handled. There are additional manuals which describe other things, such as the perl-policy which covers how Perl packages should be handled and the menu-policy which covers how menu entries should be structured.

To a complete newcomer these policies might be a little bit dense and hard to understand at first, but after working through an example or two things become much clearer. A useful reference is the Debian New Maintainer Guide which describes the creation of Debian packages with good examples.

After the package is made?

Once you have a package your job is almost complete, the software is ready and could be added to the repository. How that is done depends largely on who you are.

If you're an existing developer then uploading is a simple job, if you're not a member of the project, but intend to become one, then you must find an existing developer to check your work and "sponsor" your upload.

Sponsored Uploads

In general new developers join the Debian project by creating and having new packages uploaded to the archive. These might be either wholly new pieces of software or previously orphaned packages which have no current maintainer.

Because non-developers may not upload directly into the Debian archive such uploads must be "sponsored".

If you're not a member of the project and would like to join then you should start by becoming familiar with the New Maintainer's Corner, this describes exactly how a person goes through the process of becoming a Debian Developer.

Sidestepping that process for a moment though, if you have a package which you have previously filed an ITP bug for, and subsequently gone on to create, then your next job is to find a developer to upload it on your behalf.

The common way of doing that is to seek assistance from the debian-mentors mailing list. This list specialises in helping and "mentoring" new applicants - both in terms of assisting in the creation of packages, and in answering questions about the process.

The debian-mentors FAQ should provide a useful reference.

To find a sponsor you should send a message describing your package, and providing a link to the location from which the source can be downloaded. Nobody will be interested in merely seeing a link to the binary .deb file, because they will wish to see how your package is constructed and will rebuild it themselves to test it.

If you receive a response then you may also receive some questions, and suggestions for improving the package. Once your potential sponsor is happy with the package they will upload it for you.

(Matthew Palmer wrote a nice checklist for people sponsoring package uploads which might be useful for people to read. It highlights a couple of areas where new packagers sometimes make mistakes.

Developer Uploads

Developers have the power to directly upload their created packages. This isn't an excuse for failing to file ITP bugs, nor is it a divine right - the packages which are uploaded should still conform to the relevant policies, and still have to wait their time in the New Queue.

Developers, if they have time for it, are encouraged to join the debian-mentors mailing list and sponsor packages which interest them.

The Developer's Reference contains a section on sponsoring uploads which explains how they should be carried out, and what you need to do. This is nicely augmented by the sponsorship checklist which was previously mentioned.

The New Queue

New packages which are uploaded to the archives are not blindly added - instead they require manual addition to the archives. This is the job of the "ftp masters".

The current membership of the ftp master team, like all other teams, is listed upon the Debian Organizational Structure page.

The package which has been uploaded will sit in the NEW Queue until it is either approved and moved into the official archives - or rejected, in which case the package uploader will receive an explanation of why its addition was refused.

For reference "new packages" includes wholly new packages and packages which have had a new name - such as versioned library packages. This explains part of the delay involved in the acceptance of new packages as there are always new packages to examine and accept/reject.

What Next?

Whilst having a package in the official archives is usually the goal of a packager, or an author, once the package is accepted that's not the end. Instead that's the start of the support process.

Once the package is available to the Debian user community bugs will likely be found, or suggestions made for improvement - either in the package itself, or in the scripts, examples, or manpages which are included with it.

Any bugs found and reported will likely require a new upload to fix. If you've made an upload via a sponsor you should approach them again to see if they will repeat their upload. It's usually common for somebody to continue sponsoring a package which they've previously sponsored, unless time constraints make it difficult.

Another common source of problems for new packages is a failure to build upon some of the platforms which Debian supports - in that case the build logs can be used to spot warnings, and compilation errors.

If you have a bug-free package congratulations you can now sit back and see how many people are using it via the popularity contest - which we mentioned the Debian popularity contest briefly in the past.

 

 


Posted by Anonymous (158.125.xx.xx) on Mon 11 Jul 2005 at 17:34
Thanks! This has really made me want to package my programs for the project and maybe even become a Debian developer to maintain them (if they're accepted). best regards, Matthew Atkinson

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Mon 11 Jul 2005 at 17:44
[ View Steve's Scratchpad | View Weblogs ]

I'm glad it was useful.

There is a middle-path I didn't cover here which involves filing RFP bugs - RFP meaning 'Request For Package.

The intention is that a user of Debian can file a bug saying "I think foo" should be included, and then hopefully an existing developer will spot the bug, and package the software on their behalf.

Its use seems to have declined recently, but it's another option. Of course if you know a developer who has free time you can also mail them and say "Oooh isn't foo shiny and pretty? Wouldn't you love to package it?". ;)

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by ajt (204.193.xx.xx) on Tue 12 Jul 2005 at 12:02
[ View Weblogs ]
What is the general feeling regarding CPAN and other large software repositories?

I have a pair of modules on CPAN, and I'd be quite happy to package them up as debs, but does Debian want to encourage lots of modules within the Debian package system, or leave them in CPAN?

--
"It's Not Magic, It's Work"
Adam

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Tue 12 Jul 2005 at 12:06
[ View Steve's Scratchpad | View Weblogs ]

I'm honestly not sure, which makes this an interesting question.

There certainly are packaged CPAN modules in the archive, but what causes that I'm not sure.

My own personal preference is to use the apt system at all times, so I would rather produce a .deb file than use the CPAN system - but I'm not really sure how common that is.

I think it makes a lot of sense to have some of the framework type modules available as packages, CGI::Application, DBI, etc. For the smaller modules, especially single file modules which can be installed manually easily (eg. HTML::Scrubber) then there's less reason to package them.

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (82.192.xx.xx) on Sat 16 Jul 2005 at 08:00
Debian is well known for the software which is contained in it's repositories.

No, it's well known for the software which is contained in its repositories.

[ Parent | Reply to this comment ]

Posted by Anonymous (81.242.xx.xx) on Mon 18 Jul 2005 at 07:37
Not everybody is from English origin!

[ Parent | Reply to this comment ]

Posted by Anonymous (203.59.xx.xx) on Tue 14 Mar 2006 at 00:14
Indeed not everyone is--myself included (but I tend not to use this fact as a crutch). Steve, as far as I can tell, is British. In my experience, misusing the apostrophe to mean "look out, here comes an s" is a mistake chiefly made by those with English as their first language.

[ Parent | Reply to this comment ]

Posted by Steve (212.20.xx.xx) on Tue 14 Mar 2006 at 12:40
[ View Steve's Scratchpad | View Weblogs ]

I'm British yes, born in York (England) and currently residing in Edinburgh (Scotland).

I've made a real effort to be more careful with my writing over the past year, most of the time my errors are a result of my typing being faster than my thinking!Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (157.159.xx.xx) on Wed 22 Apr 2009 at 12:18
Where are source packages for packages which are in the new queue ? ... couldn't find this information anywhere :(

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Wed 22 Apr 2009 at 12:22
[ View Steve's Scratchpad | View Weblogs ]

Packages in the new queue aren't visible to people outside the FTP-master team.

(Otherwise we'd essentially be making them available even though they'd not been checked for legal/quality issues.)

Steve

[ Parent | Reply to this comment ]

Posted by olberger (157.159.xx.xx) on Wed 22 Apr 2009 at 13:05
At least the .diff source package may be interesting for those whishing to test/debug new proposed packages... and as such wouldn't jurt more that what can be already in Debian's VCS like alioth'git or SVN when packages sources are committed there in the first place...

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Wed 22 Apr 2009 at 14:05
[ View Steve's Scratchpad | View Weblogs ]

I agree that it would be useful - but to get this changed you'd need to get the FTP-masters to update things, and I suspect the answer would be "No".

Steve

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 149 votes ~ 0 comments )