Building Debian CD-ROMS Part 1 - dfsbuild

Posted by Steve on Sun 24 Apr 2005 at 16:25

dfsbuild is a relatively recent tool which allows you to build livecds based upon Debian. It's intended purpose is to allow you to install Debian from scratch, but because it is a simple means of building livecds it is useful for people who use Debian and don't wish to reinstall.

This is the first of two systems we're going to look at for building directly bootable, custom, Debian CD-ROMs the other being bootcd next time.

Installing the software is as simple as running:

root@mystery:~# apt-get install dfsbuild
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
  apt-move cdebootstrap libdebian-installer-extra4 libdebian-installer4
The following NEW packages will be installed:
  apt-move cdebootstrap dfsbuild libdebian-installer-extra4
  libdebian-installer4
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 1131kB of archives.
After unpacking 3316kB of additional disk space will be used.
Do you want to continue? [Y/n] 

Once the package is installed you'll find the default configuration created and placed in the file /etc/dfsbuild/dfs.cfg.

The configuration file contains details of the repositories you wish to use, for downloading software from along with details of the packages you wish to install upon your new system.

By default the image includes testing, stable, and unstable repositories and builds the image against packages from testing. I changed this by updating the configuration to read:

# Repositories to mirror.  Details about each one are configured below.
dlrepos = testing

# Repository to build the CD with.  Must be in above list.
suite = testing

This saves you from mirroring packages from systems you don't care about.

As I'm running on an x86 system I also removed all references to amd64 from the default configuration file, once that was done I had to tell the software where it should mirror the relevent packages from. This is as simple as adding a line next to the relevent repository:

[repo testing]
suite = testing
mirror = http://http.us.debian.org/debian 

The only other thing I had to do was update the following section which tells the build script which kernel to use. The kernel here comes from my local filesystem - so you'll either have to download it, or update the file:

# Debs from local fs to unpack on live FS (will not be configured)
unpackdebs = /var/cache/apt/archives/kernel-image-2.6.8-2-k7_2.6.8-13_i386.deb 

The other thing you might wish to adjust is the list of packages which are to be included in the image:

# Packages to install on live FS, on all archs, besides base system
allpackages = util-linux mkisofs cdrecord dvd+rw-tools ocaml hugs ghc6 perl
        vim nano joe libncurses5-dev ftp ssh telnet elinks less zip unzip tar 
        info man-db manpages-dev manpages cdebootstrap diff patch gawk 
        tcpdump bash devfsd module-init-tools modutils 
        rsh-client tftp traceroute iputils-tracepath strace iputils-ping iptraf
        iproute ipchains ipfwadm iptables ifupdown dhcp-client
        bind9-host whois dnsutils rsync rdiff-backup mutt netcat
        cpio buffer alien bzip2 dpkg-dev devscripts busybox-static
        dash sash pciutils hotplug discover buffer cramfsprogs minicom
        hdparm ntpdate disktype ddrescue umsdos recover dpkg-repack
        devscripts debhelper emacs21-nox mtr-tiny python-dev build-essential
        g++ wget lftp lynx cu debconf grep-dctrl lsof sysutils epic4 screen
        gnupg dfsbuild less sudo

These pacakgs are enough to test that the software works, and also have a useful system afterwards. By default the configuration file contained a few packages which weren't installable from testing, such as ext2-resize, this is another reason why my configuration file contains fewer packages than those in the default setup.

(My complete configuration file can be studied for reference).

Once the configuration file has been amended for your preferences you can then build the CD-ROM with the following command:

dfsbuild -c /etc/dfsbuild/dfs.cfg dir

The directory you give should be empty it is wiped with no prompting as the imaging creation process starts.

dfsbuild will download the named repositories and packages to place upon the CD-ROM, and then proceed to build the bootable image.

All being well the bootable image will be found in the working director you specified image.iso.

Tweaking Things

As well as just including raw packages from the Debian mirror of your choice you can also create files and directories upon the bootable image, append data to existing files, remove files, or create symbolic links.

In my configuration file you can see examples of each of these operations.

For example I create the /etc/resolve.conf file specific to my home network so that DNS lookups will work on the booted system.

Creating a file is as simple as placing the following in the configuration file:

[createfiles]
/etc/hostname = bootablesystemshostname

[createfiles]
/etc/resolve.conf = search my.flat
search 192.168.1.1

You can also append to existing files with the following:

[appendfiles]
/etc/network/interfaces = iface eth0 inet dhcp
  iface eth1 inet dhcp
  iface eth2 inet dhcp
  iface wlan0 inet dhcp
  iface wlan1 inet dhcp
  iface wlan2 inet dhcp
  iface ath0 inet dhcp

Finally symbolic links can be created as you can see in the sample configuratoin file:

[symlinks]
/etc/mtab = /proc/mounts
Testing The Image

If you have Qemu installed you can test out your new image without booting a real machine:

qemu -boot c -cdrom image.iso
What would you use it for?

If you're wondering what you'd use a bootable Debian CD-ROM for then here are a couple of suggestions:

  • Use as a rescue disk in case you render your machine unbootable.
  • Use to install Debian on a new machine.
  • Creating a booting disk which will serve as a webserver, mounting the documents its serving via NFS.
  • Acting as a proxy server, or traffic sniffer in a simple manner.
  • Testing a potentially compromised machine without disturbing it.

There are many other possible uses!

Warning: using transparent compression. This is a nonstandard Rock Ridge extension. The resulting filesystem can only be transparently read on Linux. On other operating systems you need to call mkzftree by hand to decompress the files.

 

 


Posted by asg (152.43.xx.xx) on Mon 25 Apr 2005 at 12:54
Another very useful way to build live CDs of Debian systems is the bootcd package. You can take any kernel (with or without initrd) and a chroot (or even just your existing running Debian system) and build a live CD iso in a few minutes with minimal config.

An added bonus is that the live CD can install itself the a system drive after booting making it a nice way to do custom installs; I use such a CD for installing Debian on SAN booting IBM HS20 Blade Servers.

-- asg

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Mon 25 Apr 2005 at 15:59
[ View Steve's Scratchpad | View Weblogs ]

I'm going to be covering that next - as I said in the introduction!

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by asg (24.93.xx.xx) on Mon 25 Apr 2005 at 16:02
Eeep, early morning without coffee; I completely failed to read that sentence

/moron asg

--asg

[ Parent | Reply to this comment ]

Posted by c3101 (168.209.xx.xx) on Tue 10 May 2005 at 11:14
Hi asg

Can you make your ISO available ? As that is exactly what I would like to do tomorrow, and time permitting, it would be a great help.

Thanks.

[ Parent | Reply to this comment ]

Posted by dosha (131.251.xx.xx) on Tue 31 May 2005 at 11:45
I keep getting this error:

P: Deconfiguring helper cdebootstrap-helper-makedev
Fatal error: exception Not_found

any ideas why?

[ Parent | Reply to this comment ]

Posted by Steve (82.41.xx.xx) on Tue 31 May 2005 at 17:41
[ View Steve's Scratchpad | View Weblogs ]

None at all I'm afraid :(

Steve
-- Steve.org.uk

[ Parent | Reply to this comment ]

Posted by Anonymous (83.109.xx.xx) on Mon 6 Jun 2005 at 12:34
How can i include a folder with music into this cd?

[ Parent | Reply to this comment ]

Posted by Anonymous (195.154.xx.xx) on Tue 7 Jun 2005 at 16:15
Don t miss to add cramfs support to your new kernel if you buil a new one . :)

And make a local cache system for your .deb ,dfsbuild will download debian packet every times .

you can do this with a software like apt-proxy

[ Parent | Reply to this comment ]

Posted by Anonymous (80.121.xx.xx) on Sat 11 Jun 2005 at 21:04
also do not forget to include things like ide-cd, iso9660, the RAM disk support(?) and everything else you need for booting STATICAllY into the kernel - dfsbuild will ignore your initrd ...
apt-cacher is another great tool for reducing bandwidth consumption

[ Parent | Reply to this comment ]

Posted by Anonymous (69.44.xx.xx) on Mon 17 Jul 2006 at 20:44
This is correct for 0.6.x.

As of 0.99.0 (in unstable, as of this writing), dfsbuild will support initramfs kernels.

[ Parent | Reply to this comment ]

Posted by rick (24.98.xx.xx) on Thu 16 Jun 2005 at 21:48
hrm. seems like a good plan, but I'm having issues:
#dfsbuild -c /etc/dfsbuild/dfs.cfg -w /mnt/new_image
.
.
P: Configuring helper cdebootstrap-helper-diverts
P: Configuring helper cdebootstrap-helper-makedevs
P: Unpacking package apt
P: Configuring package apt
E: Couldn't install root!
Fatal error: exception Shell.Subprocess_Error(_,0)

need some more debug info. even strace hasn't been as enlightnening as I'd hoped. But I do know I had to add stuff to cdebootstrap to handle "etch" (new testing) -- perhaps this issue is related.

[ Parent | Reply to this comment ]

Posted by Anonymous (62.99.xx.xx) on Wed 29 Jun 2005 at 10:59
you have to install cdebootstrap_0.3.5_i386.deb mkdir /usr/share/cdebootstrap/etch cp -R /usr/share/cdebootstrap/sid /usr/share/cdebootstrap/etch

[ Parent | Reply to this comment ]

Posted by Anonymous (130.207.xx.xx) on Wed 29 Jun 2005 at 11:51
indeed, cdebootstrap was patched to 0.3.5 after my original post. dfsbuild was also patched, and I finally got a working 2.6.12 dfs disk built :)

[ Parent | Reply to this comment ]

Posted by spewdemon (68.107.xx.xx) on Sat 16 Jul 2005 at 09:54
This package is broken.

The fix is simple:
edit the following file and add

Suite: etch
Config: etch

/usr/share/cdebootstrap/suites


I wasted at least six hours today looking to get this thing working. I thought I would share this with all of you so you might avoid the same frustration.

Thanks, it looks like a great tool (it is still building while I type)

SpewDemon.

[ Parent | Reply to this comment ]

Posted by Anonymous (81.255.xx.xx) on Wed 20 Jul 2005 at 16:02
I used your config file and create a boot CDROM .
When i boot dfscd can't found a device cdrom.
whereis the problem

Thanks

Philippe LEBRETON

[ Parent | Reply to this comment ]

Posted by Anonymous (84.133.xx.xx) on Tue 8 Aug 2006 at 22:44
The same to me and if I have understood correctly the prior postings (note that this lacks in the article) you have to select your own kernel compiled from source. The debian shipped binary kernels do not work unless dfsbuild is capable to merge their corresponding initrd into its own.
greetings, killerhippy

[ Parent | Reply to this comment ]

Posted by Anonymous (82.151.xx.xx) on Fri 9 Sep 2005 at 15:37
Hi.

After playing around a long time now, i fiannly managed to create a ISO with dfsbuild. I am using a stable (sarge) machine to generate the image (tried the current version of cdebootstrap and dfsbuild from testing and unstable, but did not work, also tried to run it on unstable). item:

I managed to bring it to work (write an iso) but now (with qmeu or on real machines, tried 3 different ones) the iso cannot find the cd-rom drive and the CD within it. The error line is:

Scanning (and then scanning for all devices that are mentioned in the config file)... Could not find a CD. Aborting.

So i tried the dfscd kernel parameter (by adding dfscd=/dev/hdc to grub), but i don't see any difference.

I took the 2.6.8-2-i386 kernel from debian, as i guess, it should work fine, but either way: The problem does not seem to come from the kernel.

Does anybody know what's wrong here?

thanks,

thomas (tg@nadir.net)

[ Parent | Reply to this comment ]

Posted by Anonymous (83.198.xx.xx) on Sun 12 Feb 2006 at 17:56
After having installed and launch it in debian sid, it download
several package. Then it goes wrong :
Updating from local Packages files...
cd: 1867: can't cd to /mnt/oldebian/livecd/target/var/lib/apt/lists
Unknown error: getfiles: 2.
Fatal error: exception Shell.Subprocess_error(_, 0)

obviously, it tries to go in a directory tha doesn't exist (I checked
it, nothing but "cache" in the var directory).

What's the matter ? How can I make it look in the right directory ?

[ Parent | Reply to this comment ]

Posted by Anonymous (81.151.xx.xx) on Fri 25 May 2007 at 14:04
Can anyone help please I seem to keep having problems but I am not sure why any help would be great.


debian-dev:/home/timm# dfsbuild -v -V -R -c /etc/dfsbuild/dfs.cfg -w dfs
[dfs/DEBUG] VERBOSE MODE (DEBUG) engaged.
[dfs/DEBUG] Command line parsed, results: [("v",""),("V",""),(" ;R",""),("c","/etc/dfsbuild/dfs.cfg "),("w","dfs")]
[dfs/DEBUG] Config file parsed: fromList [("DEFAULT",fromList [("allpackages","util-linux mkisofs cdrecord dvd+rw-tools perl\nvim nano joe libncurses5-dev ftp ssh telnet elinks less zip unzip tar\ninfo man-db manpages-dev manpages cdebootstrap diff patch gawk\ntcpdump bash module-init-tools modutils\nrsh-client tftp traceroute iputils-tracepath strace iputils-ping iptraf\niproute iptables ifupdown dhcp-client\nbind9-host whois dnsutils rsync rdiff-backup mutt netcat\ncpio buffer alien bzip2 dpkg-dev devscripts busybox-static\ndash sash pciutils discover buffer cramfsprogs minicom\nhdparm ntpdate disktype ddrescue recover dpkg-repack\ndevscripts debhelper emacs21-nox mtr-tiny python-dev build-essential\ng++ wget lftp lynx cu debconf grep-dctrl lsof sysutils epic4 screen\ngnupg dfsbuild less sudo nano"),("builder","John Goerzen <jgoerzen@complete.org>"),("compress"," yes"),("deletefiles","/etc/rcS.d/*discover\n/ etc/rcS.d/*lvm\n/var/log/dpkg.log\n/var/log/bootstrap.log"), ("dlrepos","stable"),("docdir",&quo t;/usr/share/doc/dfsbuild"),("dontcompress"," /boot\n/etc/*boot*"),("libdir","/usr/lib/dfsb uild"),("makedirs","/root/.elinks"),(&qu ot;mirror","http://ftp.uk.debian.org/debian"),("name","De bian From Scratch (DFS)"),("ramdisk_files","/etc\n/tmp\n/var/tm p\n/var/lock\n/var/run\n/var/state\n/root\n/var/lib/misc"),( "suite","stable"),("version"," 0.99.0")]),("appendfiles",fromList [("/etc/issue","Welcome to Debian From Scratch (DFS)\n.\nTo login, supply username \"root\" and just press Enter if asked for a\npassword."),("/etc/network/interfaces","if ace eth0 inet dhcp\niface eth1 inet dhcp\niface eth2 inet dhcp\niface wlan0 inet dhcp\niface wlan1 inet dhcp\niface wlan2 inet dhcp\niface ath0 inet dhcp"),("/root/.bashrc","export WWW_HOME=\"file:///opt/dfsruntime/home.html\"\ndfshints")]),("createfiles",fromList [("/etc/hostname","dfs"),("/etc/syslog.c onf","*.* /dev/tty8\n*.info /dev/tty7")]),("i386",fromList [("bootloader","grub-no-emul"),("dlrepos ","stable"),("packages","%(allpacka ges)s grub read-edid linux-image-amd64")]),("symlinks",fromList [("/etc/mtab","/proc/mounts")])]
[dfs/DEBUG] Working dir is dfs
[dfs/DEBUG] Working dir created
[dfs/DEBUG] Initial cwd is /home/timm
[MissingH.Cmd.pipeFrom/DEBUG] dpkg ["--print-architecture"]
[dfs/INFO] Welcome to dfsbuild. Image architecture: "i386"
[dfs/DEBUG] dfsbuild is running as root.
[dfs/INFO] Using working directory /home/timm/dfs
[dfs/INFO] Using library directory /usr/lib/dfsbuild
[dfs/INFO] Running.
[dfs/DEBUG] Processing at state InitPrepped
[dfs/INFO] Preparing directory for ramdisk...
[dfs/CRITICAL] Exception: /home/timm/dfs/target/tmp/busybox: createDirectory: already exists (File exists)
dfsbuild: /home/timm/dfs/target/tmp/busybox: createDirectory: already exists (File exists)

[ Parent | Reply to this comment ]

Posted by Anonymous (85.198.xx.xx) on Sat 11 Aug 2007 at 14:54
You must delete ${DIR}/mirror directory.Then run dfsbuild.

[ Parent | Reply to this comment ]

Posted by Anonymous (85.198.xx.xx) on Wed 8 Aug 2007 at 12:51
Dear Steave,
I'm using your cfg file.Of course, thank you for your white paper.
dfsbuild can't accept me everything that enclosed with [].It say pharse error at same line.
Do you have an idea?
--Mohsen

[ Parent | Reply to this comment ]

Posted by Anonymous (195.117.xx.xx) on Thu 6 Sep 2007 at 14:06
I have just managed to make dfsbuild work. If you try it on etch you have to edit the list of packages on the list in dfs.cfg. If a package isn't available or can't be installed dfsbuild fails with a VERY misleading message. You have to make sure that all the packages are available first.

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 124 votes ~ 0 comments )