Making Debian packages from proprietary software

Posted by mwr on Wed 30 May 2007 at 06:35

Tags:

One of my main goals for a managed infrastructure is to make sure I have consistent versions of end-user applications installed everywhere. My users aren't too picky about the version of xemacs installed, but they've got pretty stringent requirements on having a particular version of ANSYS, Abaqus, Fluent, Maple, Matlab, and other large non-free/no-source-available software packages. And they don't all agree on which version should be loaded, so I keep several loaded at once.

Previously, I'd install the software on one candidate system, test it, rsync it to the other systems that needed it, and hopefully put it in the appropriate system image later. And the nightly rsyncs on my dual-boot labs were a pain (can't rsync multiple repositories to one target, so let's have 42 clients simultaneously rsync two NFS-mounted trees to their local disks). So all things considered, it was time to make Debian-style packages of the big, expensive third-party stuff.

Goals (using Matlab as an example):

  1. The ability to run 'apt-get install matlab74', and get the latest packaged version of Matlab 7.4.
  2. The ability to keep Matlab 7.4, Matlab 7.2, and other versions loaded simultaneously.
  3. The ability to run any version of Matlab currently loaded on-demand -- if I type 'matlab74', I expect Matlab 7.4. If I type 'matlab72', I expect Matlab 7.2.
  4. For the less picky users, 'matlab' with no numbers attached should still run some version of Matlab.
  5. I need to use the same "source" tree for both i386 and amd64 packages. I don't want to duplicate my packaging efforts.
More details on how I'm doing this after the jump.From the Debianized source directory (naming convention packageMM-NN+L: where MM is the abbreviated major and minor revision of the package; NN is a Debian-style version number based off the vendor's build information, and L is the local packaging revision):
ch226-21:~/matlab74-7.4.0.287+0.3# ls -F
debian/  Makefile  src/
ch226-21:~/matlab74-7.4.0.287+0.3# ls -F src/*
src/amd64:
opt/  usr/

src/i386:
opt/  usr/
ch226-21:~/matlab74-7.4.0.287+0.3# ls -F src/amd64/opt/matlab src/amd64/usr/bin/
src/amd64/opt/matlab:
7.4-r2007a/

src/amd64/usr/bin/:
matlab74@  mbuild74@  mcc74@  mex74@
ch226-21:~/matlab74-7.4.0.287+0.3#
So the src directory contains one folder per Debian architecture this package is available for (goal 5). Each of those architecture folders contains a version-specific software folder under opt. I'm trying to standardize my initial installations to /opt/packagename/version -- many commercial packages, including ANSYS, Abaqus, and Fluent, already default to this style of installation. And so far, the rest of the packages I've worked in will accept this style of installation without complaint. This takes care of part of goal 2 above.

The usr/bin directory contains relative symlinks for the executables people expect to have available. All of these links are tagged with a version. This takes care of goal 3 above.

ch226-21:~/matlab74-7.4.0.287+0.3# ls -l src/amd64/usr/bin/
total 0
lrwxrwxrwx 1 root root 38 2007-05-28 10:31 matlab74 -> ../../opt/matlab/7.4-r2007a/bin/matlab
lrwxrwxrwx 1 root root 38 2007-05-28 10:31 mbuild74 -> ../../opt/matlab/7.4-r2007a/bin/mbuild
lrwxrwxrwx 1 root root 35 2007-05-28 10:31 mcc74 -> ../../opt/matlab/7.4-r2007a/bin/mcc
lrwxrwxrwx 1 root root 35 2007-05-28 10:31 mex74 -> ../../opt/matlab/7.4-r2007a/bin/mex
ch226-21:~/matlab74-7.4.0.287+0.3#
Now let's look at the top-level Makefile:
build:
        @echo "No building required. Binary package."

install:
        cp -av src/`dpkg --print-architecture`/* ${DESTDIR}

clean:
        @echo "Cleaning not required. Binary package."
Quite possibly the simplest Debian Makefile ever. All we have to do is copy over the desired architecture's directory tree to its proper destination. Now for the contents of the debian directory:
ch226-21:~/matlab74-7.4.0.287+0.3# ls debian/
changelog  control    dirs  postinst  README
compat     copyright  docs  prerm     rules
ch226-21:~/matlab74-7.4.0.287+0.3#
The changelog and compat files are typical for any Debian package. The copyright file contains the software license agreement from the Mathworks, authors of Matlab. The dirs file contains one entry for /opt/matlab/7.4-r2007a. The docs file is empty. The README file is a simple statement that I repackaged Matlab 7.4-r2007a for the TTU Computer-Aided Engineering Network.

As for the more complicated stuff, rules was generated by dh_make for a single binary package, and I commented out the dh_strip and dh_shlibdeps lines. I tended to get errors with these included. Lots of commercial software doesn't have much in the way of dependencies. About as often, they include a version of a library that conflicts with what's currently installed (libgcc seems to be a common problem lately). Part of my testing is running the software and seeing if there were any missing libraries, so I'll just add those to the control file myself. As for control itself, it's pretty simple:

Source: matlab74
Section: unknown
Priority: extra
Maintainer: Mike Renfro <renfro@tntech.edu>
Build-Depends: debhelper (>= 5)
Standards-Version: 3.7.2

Package: matlab74
Architecture: any
Depends: libxext6, libxp6, libxt6
Description: MATLAB - The Language of Technical Computing
 MATLAB is a high-level language and interactive environment that enables
 you to perform computationally intensive tasks faster than with traditional
 programming languages such as C, C++, and Fortran.
 .
 Packaged for TTU Computer-Aided Engineering Network. Includes the following:
 .
 MATLAB, Simulink, Communications Blockset, Communications Toolbox, Control
 System Toolbox, Distributed Computing Toolbox, Fuzzy Logic Toolbox, Genetic
 Algorithm and Direct Search Toolbox, Image Processing Toolbox, MATLAB
 Compiler, Neural Network Toolbox, Optimization Toolbox, Partial Differential
 Equation Toolbox, Real-Time Workshop, Robust Control Toolbox, Signal
 Processing Blockset, Signal Processing Toolbox, SimPowerSystems, Simulink
 Control Design, Stateflow, Statistics Toolbox, Symbolic Math Toolbox, System
 Identification Toolbox, Wavelet Toolbox.
As to the final pieces, postinst and prerm, these both need calls to update-alternatives (Steven Kroon has a good writeup on update-alternatives I worked from). This way, I can have a sensible default version of Matlab available as 'matlab', and let people pick what version they want by specifying it, e.g. 'matlab74' (goal 4 above). For postinst, the configure case has:
    configure)
        update-alternatives --install \
            /usr/bin/matlab matlab /usr/bin/matlab74 300 \
            --slave /usr/bin/mbuild mbuild /usr/bin/mbuild74 \
            --slave /usr/bin/mcc mcc /usr/bin/mcc74 \
            --slave /usr/bin/mex mex /usr/bin/mex74
    ;;
As for prerm, the primary case has:
    remove|upgrade|deconfigure)
        update-alternatives --remove matlab /opt/matlab/7.4-r2007a/bin/matlab
    ;;
And that's it. Running dpkg-buildpackage from the top-level folder makes one ginormous .deb package. I mean seriously large. For Matlab, the .deb files each weigh in at around 800 MB. The source archive is 1.6 GB, expanding out to 2.9 GB. But storing those in one central backed-up location instead of once per backed-up system image is worth it.

Ah, I forgot about goal 1. So far, we've got a package we can run through dpkg -i, but we still need to get it into an apt-compatible repository. That part warrants another post entirely (and if that one's still inaccessible for you, and Steve is ok with it, I'll cross-post it here, too).

This article is reposted from my website, which is apparently inaccessible to some portion of the world.

 

 


Posted by mwr (149.149.xx.xx) on Wed 30 May 2007 at 16:03
[ View Weblogs ]
Typo in the postinst excerpt: only one backslash is needed per line, instead of two.

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Thu 31 May 2007 at 09:22
[ View Steve's Scratchpad | View Weblogs ]

Fixed.

If you spot any more errors feel free to click the "Edit Article" link on the right hand side.

Steve

[ Parent | Reply to this comment ]

Posted by cparker (204.136.xx.xx) on Tue 5 Jun 2007 at 23:51
[ View cparker's Scratchpad | View Weblogs ]
Slightly off-topic: The title of this article really should read "Making Debian packages from proprietary software". We all should know by now that free (libre) software can be commercial.

[ Parent | Reply to this comment ]

Posted by mwr (149.149.xx.xx) on Tue 5 Jun 2007 at 23:55
[ View Weblogs ]
Fixed. I assume given the number of reads this has gotten, and the absence of technical errors pointed out, that my instructions are either perfect or entirely inapplicable to everyone else here.

[ Parent | Reply to this comment ]

Posted by Anonymous (216.124.xx.xx) on Wed 6 Jun 2007 at 15:46
Its a great article! I've always meant to get into packaging more...

However its one of those articles that I can't just follow step-by-step and test it out in 5 minutes. So if you do have technical errors, expect some comments to show up in a few weeks or months. :)

Michael Schurter

[ Parent | Reply to this comment ]

Posted by jparrella (65.254.xx.xx) on Wed 13 Jun 2007 at 22:13
[ View Weblogs ]
Yes. This is a very helpful post, since lots of organizations using Debian would like to keep Linux-native proprietary applications integrated into their in-house repositories. I've seen the SAP client, sprotectlinux, ies4linux and lots of other proprietary software in .deb format. Note, however, that some licenses may restrict further redistribution in the form of a "package" when it doesn't flash an end user license agreement or something like that. In that cases, the sun-java5-plugin example is a good one: preseed the answers for a debconf template asking to accept the license :D

[ Parent | Reply to this comment ]

Posted by Dosenpfand (128.131.xx.xx) on Mon 26 May 2014 at 09:55
Working link for the writeup on the alternatives system: https://web.archive.org/web/20090525175833/http://blog.stevenkroo n.com/2006/08/29/debian-update-alternatives/
(Since I can't edit the article)

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 459 votes ~ 5 comments )