Setting up a server for PXE network booting
Posted by Steve on Fri 22 Dec 2006 at 09:18
If you're looking to perform a lot of system recovery, or system installation, then network booting with PXE is ideal. PXE allows you to boot up a system and have it automatically get an IP address via DHCP and start booting a kernel over the network.
PXE itself stands for "Pre-boot eXecution Environment", which describes how it works in the sense that the clients using it haven't booted in a traditional manner.
In order to use PXE you need to setup a boot-server which will allow client systems to :
- Request an IP address (via DHCP)
- Download a kernel (via TFTP)
With both of these services in place any system which supports PXE/network-booting (you might need to enable it in the BIOS) should be able to gain an IP address, fetch a kernel, and boot without an installed operating system.
(This is ideal for systems which can't be booted by a traditional approach; for example your new AMD-64 system which doesn't have a CD/DVD drive!)
Our SetupTFTP SetupFor the purposes of this article we'll assume:
- We're working with a small network 192.168.1.0/24
- We'll allow all local machines to boot and get an IP address via DHCP from the range 196.168.1.70-192.168.1.100
- Our "boot-server" is the host "itchy" with IP address 192.168.1.50
- We will serve the same kernel to each host.
In our example we'll configure a PXE-server which will allow remote systems to run the Debian Etch installer, but nothing here is specific to that. PXE allows you to configure a system to boot from an arbitrary kernel (and matching ramdisk if you wish to use one). With the correct configuration you can even cause the clients to mount a remote file-system via NFS and have a diskless thin-client system.
DHCP SetupTFTP is a very simple file-transfer protocol, which is very similar to FTP but which doesn't use any kind of authentication. If you're not already running a TFTP server you can install one by running:
root@itchy:~# apt-get install tftpd-hpaOnce installed you'll need to enable it by editing the file /etc/default/tftpd-hpa. You should change RUN_DAEMON to yes, leaving you with contents like these:
#Defaults for tftpd-hpa RUN_DAEMON="yes" OPTIONS="-l -s /var/lib/tftpboot"Now create the root directory, if it is missing, and start the server:
root@itchy:~# mkdir -p /var/lib/tftpboot root@itchy:~# /etc/init.d/tftpd-hpa start Starting HPA's tftpd: in.tftpd.Once our systems have retrieved an IP address via DHCP they will request files from beneath the /var/lib/tftpboot root directory. We'll come back to the contents of this directory shortly.
PXE ConfigurationIf you don't already have a DHCP server configured upon your LAN you'll need to install one. If you're using a small home router, or similar, to provide local DHCP services you must disable this first. (Since we require the DHCP server to pass back some extra options to clients which the majority of routers won't allow).
Discussing a full DHCP installation is mostly beyond the scope of this introduction but the thing we're trying to do is fairly simple. The goal of the DHCP server in this setup is twofold:
- We obviously want to use it to allow clients to request and receive an IP address.
- We want to cause the DHCP "answer" to give some extra details to the clients which are requesting an address:
- The address of the TFTP server.
- The initial filename to request from the TFTP server.
The most common DHCP server is the dhcp-server package, and you can install this by running:
root@itchy:~# apt-get install dhcp3-serverOnce installed the server is configured in the file /etc/dhcp3/dhcpd.conf, and there are a lot of available options described there. For our example we'll use the following configuration:
option domain-name-servers 62.31.64.39, 62.31.112.39; default-lease-time 86400; max-lease-time 604800; authoritative; subnet 192.168.1.0 netmask 255.255.255.0 { range 192.168.1.70 192.168.1.100; filename "pxelinux.0"; next-server 192.168.1.50; option subnet-mask 255.255.255.0; option broadcast-address 192.168.1.255; option routers 192.168.1.1; }Here we've configured the server to give out IP addresses from the range 192.168.1.70-100, set the default gateway to be 192.168.1.1 and use our ISP's nameservers.
We've also used next-server to point to the TFTP server we're using (which is the same host as our DHCP server, but doesn't need to be). We've chosen the default name of pxelinux.0 as the name of the file for booting clients to request.
Using dnsmasq instead
Personally I use the dnsmasq package to provide DHCP services upon my LAN, since this is small and simple and provides other useful abilities, setting up PXE booting with dnsmasq just requires the addition of the following line to /etc/dnsmasq.conf:
dhcp-boot=pxelinux.0,itchy,192.168.1.50(Again we've setup the filename along with the name and IP address of the TFTP server which is "itchy" / 192.168.1.50 in this example)
Restarting the service after this change is as simple as:
root@itchy:~# /etc/init.d/dnsmasq restart Restarting DNS forwarder and DHCP server: dnsmasq.
Sample RunNow that we've configured the TFTP and DHCP servers we need to go back and complete the configuration. By default when a client boots up it will use its own MAC address to specify which configuration file to read - however after trying several options it will fall back to requesting a default file.
We need to create that that file, which will contain the list of kernels which are available to boot, we'll firstly need to create a directory to hold it:
root@itchy:~# mkdir /var/lib/tftpboot/pxelinux.cfgNow save the following as /var/lib/tftpboot/pxelinux.cfg/default:
DISPLAY boot.txt DEFAULT etch_i386_install LABEL etch_i386_install kernel debian/etch/i386/linux append vga=normal initrd=debian/etch/i386/initrd.gz -- LABEL etch_i386_linux kernel debian/etch/i386/linux append vga=normal initrd=debian/etch/i386/initrd.gz -- LABEL etch_i386_expert kernel debian/etch/i386/linux append priority=low vga=normal initrd=debian/etch/i386/initrd.gz -- LABEL etch_i386_rescue kernel debian/etch/i386/linux append vga=normal initrd=debian/etch/i386/initrd.gz rescue/enable=true -- PROMPT 1 TIMEOUT 0This file instructs the client to display the contents of the file boot.txt so create that too:
- Boot Menu - ============= etch_i386_install etch_i386_linux etch_i386_expert etch_i386_rescueThe only remaining job is to download the official Etch installer kernel and associated files and save them in the directories specified in the default file we created:
root@itchy:~# cd /var/lib/tftpboot/ root@itchy:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/pxelinux.0 root@itchy:~# mkdir -p /var/lib/tftpboot/debian/etch/i386 root@itchy:~# cd /var/lib/tftpboot/debian/etch/i386 root@itchy:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/linux root@itchy:~# wget http://ftp.uk.debian.org/debian/dists/etch/main/installer-i386/current/images/netboot/debian-installer/i386/initrd.gzWhen these commands have been completed we'll have the following structure:
root@itchy:~# tree /var/lib/tftpboot/ /var/lib/tftpboot/ |-- boot.txt |-- debian | `-- etch | `-- i386 | |-- initrd.gz | `-- linux |-- pxelinux.0 `-- pxelinux.cfg `-- default 4 directories, 5 files(We only used debian/etch here in case we want to offer other installers in the future. You can put everything in one directory if you wish, just update pxelinux.cfg/default to match.)
We should now be ready to test the setup.
We've installed a pxelinux.0 file which will instruct booting clients to request pxelinux.cfg/default. This will then make a list of boot options available, which are displayed by the simple boot menu file we created.
The files which are used for booting are stored beneath the TFTP root directory and thus accessible to booting clients.
A sample remote boot looks like this:
PXE entry point found (we hope) at 9AE5:00D6 My IP address seems to be C0A80146 192.168.1.70 FTFTP prefix: Trying to load: pxelinux.cfg/01-00-14-22-a1-53-85 Trying to load: pxelinux.cfg/C0A80146 Trying to load: pxelinux.cfg/C0A8014 Trying to load: pxelinux.cfg/C0A801 Trying to load: pxelinux.cfg/C0A80 Trying to load: pxelinux.cfg/C0A8 Trying to load: pxelinux.cfg/C0A Trying to load: pxelinux.cfg/C0 Trying to load: pxelinux.cfg/C Trying to load: pxelinux.cfg/default - Boot Menu - ============= etch_i386_install etch_i386_linux etch_i386_expert etch_i386_rescueAs you can see the system here attempted to load several configuration files, based upon its MAC address (01-00-14-22-a1-53-85) initially then falling back to the octets in the IP address it was given by DHCP (192.167.1.70).
Finally it managed to load a working configuration using the last-chance default file we created. This in turn instructed it to show the boot menu we created.
From here on the system will boot into whichever kernel you specify. (We could configure the system to timeout here and just boot into a default option, but we didn't.)
From here on you should understand how PXE can be used to boot an arbitrary kernel and initial ramdisk. Later we'll look at mounting a remote file-system over NFS to provide a diskless thin-client.
I've been playing with this the last 2 days because I got 2 soekris boards.
The setup you provide here offers PXEboot to all machines in the subnet.
What I want is to provide PXEboot options to specific hosts only.
I managed to get this done in dhcp3-server but I want to do it with dnsmasq (because now I have to connect the soekris to a small switch and use an old laptop for temp dhcp/tftp server)
Do you happen to know how to do this with dnsmasq ?
(in dhcp3-server I used a host "bla" {} block)
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Looking at the documentation I can't see an obvious way to do this - it seems to be an all or nothing setting. There are other flags in the dnsmasq.conf file for setting specific DHCP options for hosts by ip/name/mac address but I can't see how to combine these.
The way I'd do it is to broadcast globally, but disable network booting for the other hosts on the LAN, or just press "Esc" to cancel the netboot PROM.. (That works well for me since I rarely reboot!)
[ Parent | Reply to this comment ]
A lot of options can be prepanded with some identifier, but not the pxeboot function.
I think I'll stick with the seperate net+laptop to bootstrap my soekris machines.
It's only to install OpenBSD on those boards + CF card so after initial setup I dont need the pxeboot anymore.
Thanks for your time.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
If you did want to keep the bootserver around for future use you could always just cause it to automatically boot from the local disk after a short timeout? That would just need your pxelinux.cfg/default file to look like this:
timeout 120
default local
prompt 1
LABEL local
DISPLAY boot.txt
localboot 0
LABEL etch_i386_install
kernel debian/etch/i386/linux
append vga=normal initrd=debian/etch/i386/initrd.gz --
...
...
[ Parent | Reply to this comment ]
I'm not familiar with the pxelinux image and couldn't get it to load the OpenBSD installer on my soekris.
I installed dhcp3-server on my debian box and disabled it in rc2.d.
Now if I need to netboot a soekris I put a static host stanza in /etc/dhcp3/dhcpd.conf, stop dnsmasq, start dhcp3-server, boot soekris, wait for the OpenBSD install to start and reverse everything.
Annoying, but it works.
[ Parent | Reply to this comment ]
dhcp-host =00:XX:XX:XX:XX:XX,net:ares,192.168.2.3,ares
dhcp-option =ares,17,192.168.2.2:/mnt/ares
dhcp-boot =net:ares,pxelinux.0,apollo,192.168.2.2
The options are a bit inconsistent (net: is not allowed for dhcp-option), but it works as documented.
[ Parent | Reply to this comment ]
[ Send Message | View dopehouse's Scratchpad ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
next-server 192.168.1.50HTH,
-- Andreas
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Parent | Reply to this comment ]
root@itchy:~# mkdir -p /var/lib/tftpboot/debian/etch/i386
root@itchy:~# cd /var/lib/tftpboot/pxelinux.cfg/etch/i386
first you create a directory and then you cd to another?
Shouldn't the last line be?
root@itchy:~# cd /var/lib/tftpboot/debian/etch/i386
or am I missing something here?
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
My impression was that this wouldn't work since the bootloader users TFTP to transfer files - and that uses udp which the router wouldn't forward ...
[ Parent | Reply to this comment ]
Shouldn't the line "range 192.168.0.70 192.168.0.100;" be "range 192.168.1.70 192.168.1.100;"?
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
[ Parent | Reply to this comment ]
Very useful for me and my project involving Thinstation.
One extra point: dnsmasq is also TFTP capable since version 2.36. You only need to add these lines in dnsmasq.conf
enable-tftp
tftp-root=/var/lib/tfptboot
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Very useful to know, thanks a lot!
[ Parent | Reply to this comment ]
I did however end up with boot.txt in the wrong directory following the instructions - it took me a while to spot it in the directory structure diagram.
Might be good to explicitly specify dropping a dir before creating boot.txt.
But it works pretty fast ;)
[ Parent | Reply to this comment ]
or
How can you create your own distro with your own stuff on it (for recovery purpose) and boot up with that image?
[ Parent | Reply to this comment ]
I have read that it is not possible to directly load a cdrom ISO because its in their nature to communicate directly ith the hardware, but is there some other way around it? Like, if I PXE boot to some virtualized mini OS, then have this load up the cdrom ISO's. Maybe something totally different from that, I'm not really sure where to look anymore, and I've been looking for nearly 15 man hours (seriously).
Any help would be greatly appreciated!
[ Parent | Reply to this comment ]
# Update BIOS via ISO image
DEFAULT dos
LABEL dos
KERNEL memdisk
append initrd=cdrom.img
The cdrom.img filename is just the ISO renamed.
I use this to network boot firmware update CDs, BMC config CDs, etc. for a Linux cluster. Works great for remote administration. As long as the KVM or serial port is behaving there's no reason to be physically present. :-)
[ Parent | Reply to this comment ]
I can easly mount the nfs if I use the mount-command on a running computer though.
I have a gateway that only runs the dhcpserver, and then I have another computer that runs the rest.
Debian install iso, memtest and the debian boots up perfectly but I cant get the nfs-support to work properly, and for that reason it doesn´t really work.
Has anyone else had the same problem?
I can mount the nfs with any ip on the subnet.
[ Parent | Reply to this comment ]
Here is much more simple:
#cd /var/lib/tftpboot/
#wget http://http.us.debian.org/debian/dists/etch/main/installer-i386/c urrent/images/netboot/netboot.tar.gz
#tar xzvf netboot.tar.gz
after this without any manual files editing you will get standard Debian install splash screen which will install Etch.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Great article. Saw two more typos though.
This:
"As you can see the system here attempted to load several configuration files, based upon its MAC address (01-00-14-22-a1-53-85) initially then falling back to the octets in the IP address it was given by DHCP (192.167.1.70)." should probably have the IP adress 192.168.1.170 and not 192.167.1.70.
And this:
We'll allow all local machines to boot and get an IP address via DHCP from the range 196.168.1.70-192.168.1.100"
Should probably have the IP adress range 192.168.1.170-192.168.1.100 and not 196.168.1.70-192.168.1.100.
Keep up the good work with the articles :)
[ Parent | Reply to this comment ]
here is the syslog
"
dhcpd: Wrote 2 leases to leases file.
dhcpd: No subnet declaration for eth0 (0.0.0.0).
dhcpd: ** Ignoring requests on eth0. If this is not what
dhcpd: you want, please write a subnet declaration
dhcpd: in your dhcpd.conf file for the network segment
dhcpd: to which interface eth0 is attached. **
dhcpd: Not configured to listen on any interfaces!
"
[ Parent | Reply to this comment ]
In addition to pxelinux.0, you must also copy "menu.c32" from /usr/lib/syslinux into "/var/lib/tftpboot" and add "DEFAULT menu.c32" to top of menu. This is a recent change as of Syslinux 3.31 and not shown in most examples on the net (and not even in an April 2008 Linux Journal story on PXE menus.)
[ Parent | Reply to this comment ]
I have tried a few, such as 67 Bootfile-Name and 150 TFTP server address without luck. I'm trying to configure our sonicwall 2040 dhcp server to work with this.
Thanks
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
It really helped me since now I don't have to look for a cd drive to install debian on some machines.
But I have 2 questions:
1.
LABEL etch_i386_install
kernel debian/etch/i386/linux
append vga=normal initrd=debian/etch/i386/initrd.gz --
LABEL etch_i386_linux
kernel debian/etch/i386/linux
append vga=normal initrd=debian/etch/i386/initrd.gz --
Why do you have this in twice, it only has a different label, is it really necessary ?
2. What would the append line for an expertgui installation look like?
Thanks.
Greetings.
Markus
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
To answer your questions:
1. I just happen to have two copies - you only need one, as you guessed.
2. I'm afraid I'm not sure.
[ Parent | Reply to this comment ]
So where can I find this article because I am strugling with mounting the thin clients over NFS?
[ Parent | Reply to this comment ]