A brief introduction to xen-tools

Posted by Steve on Sat 16 Jun 2007 at 13:20

Tags: , , , ,

It is no secret that I'm a big fan of the open source Xen virtual machine hypervisor, and I've written several tools to make using it under Debian GNU/Linux more straightforward. Here we'll take a quick look at using xen-tools to easily create new Xen guest domains.

Overview of xen & xen-tools

The Xen system consists of an open-source hypervisor which allows a single host machine to execute multiple virtual machines at the same time. These virtual machines, or "guest domains" in Xen-speak, look and feel identical to real machines, complete with networking etc.

In order to get started with Xen you must first install it, but once it is up and running the next job becomes that of installing new virtual machine(s) upon your host. This is the problem that the xen-tools package was created to solve.

The xen-tools package

The xen-tools package consists of a collection of small perl-scripts. The ones used most frequently are:

xen-create-image

This script allows you to easily create a new Xen guest domain. The new guest will be configured with networking, along with an installation of openssh.

xen-list-images

This script allows you to see which guests you have installed, along with their networking details (if the guests are not configured using DHCP).

xen-delete-image

This script allows you to delete a guest domain which you've previously created.

Each of these scripts comes complete with an extensive man-page, and each tool reads the same file, /etc/xen-tools/xen-tools.conf, for configuration directives.

Getting Started with Xen

To get started using xen-tools you'll first of all need to have a working Xen installation upon your machine. We've had several articles about installing Xen upon Debian GNU/Linux in the past which should help you there.

Once you've got a Xen host running you'll find that you have a new command xm which allows you to interface with your Xen system. The xm command is the one you'll use to start, stop, list, and connect to your running Xen guests (once you have them installed!)

Listing Domains

The first command you'll probably want to learn is "xm list". This lists the running domains; both the guests we'll install shortly and the host system (domain0 in xen-speak).

On a plain system running Xen with no guests you'd expect to see something like this:

mine:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      675     2 r-----  53925.5

This will only show Domain-0, the host system, since we have no guests installed or running. The output also includes details such as the amount of memory this domain has, the amount of time it has been running, & etc.

Once you've created and started a couple of domains you'll see output similar to this:

mine:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0      675     2 r-----  53930.3
cfmaster.my.flat                           6       64     1 -b----    251.8
etch-builder.my.flat                       8      128     1 -b----    387.5
sarge-builder.my.flat                      9      128     1 -b----    199.0

Here you see that we have three guest domains running cfmaster.my.flat, etch-builder.my.flat, and sarge-builder.my.flat.

It's worth noticing that every single Xen guest has a name and ID associated with it. By convention your Xen guest domains will be named after the hostname of the guests - but this is not mandatory.

Shutting Down Domains

To shutdown, or reboot, a running guest you'd run either "xm shutdown [name]" or "xm reboot [name]" respectively.

So, for example, to shutdown the guest sarge-builder.my.flat we'd execute:

mine:~# xm shutdown sarge-builder.my.flat

A short while later you could run "xm list" again and see that the domain has gone from our listing.

Note: here we're shutting down the guest from the host system - if you wanted you could ssh into the running system and run the standard "/sbin/shutdown .." command.

Serial Console Access

When you have a guest installed it will be available for access over a virtual serial console - regardless of its network setup.

To connect to this serial console you need to run "xm console [name]". This will allow you to interact with your system. Exit the console by pressing "^]" - the same character you'd use to escape from a running telnet session.

Booting A Guest Domain

The last command we'll introduce is the most important - the command to start an existing guest running.

The name of this command is unfortunately a little bit misleading, rather than "bootup", or similar, the command is "xm create [filename] [-c]".

The creation command will read the name of the file which is specified from the directory /etc/xen and create the new guest domain with the machine running within it.

Usually each machine would have a file /etc/xen/hostname.domain.cfg to contain the configuration settings for it. So to start the machine cfmaster.my.flat I'd run:

root:~# xm create cfmaster.my.flat.cfg

This will load the details from /etc/xen/cfmaster.my.flat.cfg and start the machine booting. Add the -c flag to the create call if you'd like to immediately connect to the serial console and see the bootup messages.

Of course before you can boot an existing machine you must first create it - and this is where xen-tools come into play.

Getting Started with xen-tools

Upon a Debian GNU/Linux machine, running etch or higher, the only thing that you should need to do is run:

mine:~# apt-get install xen-tools xen-shell

Once installed each of the named scripts we introduced earlier should be installed.

To get started you'll need to configure the software by editing the configuration file /etc/xen-tools/xen-tools.conf.

The file should be commented fairly well, and you'll only need to make three changes to the default:

Storage Options

Xen can run domains from loopback files, physical partitions, or LVM. If you're using LVM already it is recommended you use that. If you are uncomment the line:

lvm = vain-vol

(Here "vain-vol" is the name of my volume group.)

If you're not using LVM then the next option to try is using plain loopback files, simply specify a directory to locate them in with:

dir = /home/xen

Each new guest will have two files created for it disk.img for the disk, and swap.img for swap usage.

Networking Options

We want the xen guests we create to be networked, so we'll setup the following values in xen-tools.conf:

gateway   = 192.168.1.1
netmask   = 255.255.255.0
broadcast = 192.168.1.255

The actual address of each guest we'll supply upon the command-line, but here we configure the rest of the details such that the new guests will be reachable upon our internal 192.168.1.0/24 network.

Mirror Options

The creation command will allow you to specify several different installation methods. The most common is to install guests via debootstrap. To configure this you'll need to add:

install-method = debootstrap
mirror         = http://ftp.uk.debian.org/debian/

(Obviously pick a location close to you!)

Using the Debian version of debootstrap you'll be able to install new guests of Sarge, Etch, Lenny, or Unstable. If you install the Ubuntu version of debootstrap you'll also be able to install Dapper, Edgy, Feisty, etc. (Or you could use a patched version of debootstrap for Debian.)

With these main options configured you should now be ready to proceed with the creation of your first guest. There are more options you can specify, such as whether new machines should boot automatically post-creation, or whether you should be prompted for a new root password, but you can discover those yourself.

Your First Guest

To create a guest you'll need to specify three pieces of information:

  • The distribution to install: Etch, Sarge, Edgy, etc.
  • The name of the new machine.
  • The IP address of the new machine.

As an example here I'll create a new machine called "test.my.flat" running Etch and with an IP address of 192.168.1.100:

root@vain:~# xen-create-image --hostname test.my.flat \
                  --ip 192.168.1.202 \
                  --dist etch

This will produce output showing the creation progress, as well as a complete log located at /var/log/xen-tools/test.my.flat.log:

General Information
--------------------
Hostname       :  test.my.flat
Distribution   :  etch
Fileystem Type :  ext3

Size Information
----------------
Image size     :  4Gb
Swap size      :  128Mb
Image type     :  full
Memory size    :  128Mb
Kernel path    :  /boot/vmlinuz-2.6.18-xen
Initrd path    :  /boot/initrd.img-2.6.18-xen

Networking Information
----------------------
IP Address 1   : 192.168.1.202
Netmask        : 255.255.255.0
Broadcast      : 192.168.1.255
Gateway        : 192.168.1.1


Creating ext3 filesystem on /dev/vain-vol/test.my.flat-disk
Done
Installation method: debootstrap
...
...
Advanced Usage

Rather than specifying the IP address of each new guest manually you can use a simple incrementing scheme. The system will start with a specified IP address and increment it once for each creation.

For example to start each future xen guest from the 192.168.1.200 and move upwards add the following to xen-tools.conf:

ip = 192.168.1

Now add the last octet to the file /etc/xen-tools/ips:

echo 200 > /etc/xen-tools/ips

Now the next time you create an instance the IP address of 192.168.1.200 will be used, and afterwards the 200 will be incremented in the ips file.

As an example of a simpler creation we can now run:

root@vain:~# xen-create-image --hostname=foo.my.flat --dist=sarge
Further Information

Further information on the software can be found on the xen-tools homepage.

Since I wrote the software I've been a little reluctant to post this here, but having seen some very poor guides I felt it would be useful. I tried to ensure no bias, but feel free to complain if you believe any is present. After all my code is perfect ... ;)

 

 


Posted by wuzzeb (64.5.xx.xx) on Sun 17 Jun 2007 at 01:19
Nice introduction to using Xen Tools. But I wonder if you have looked into libvirt and virt-manager at all? I've switched over from Xen to using KVM, but the nice thing about libvirt is the same program can configure xen, qemu, qemu + kqemu, and kvm.

I've sort of looked into libvirt but haven't switched yet (I still start KVM from a shell script that passes all the right arguments).

http://libvirt.org/
http://virt-manager.et.redhat.com/

John

[ Parent | Reply to this comment ]

Posted by lters (12.162.xx.xx) on Fri 29 Jun 2007 at 12:21
[ View lters's Scratchpad | View Weblogs ]
Thanks for the article Steve.

I decided to try xen and xen-tools and have a couple questions...

My test platform is Etch.
I installed Sarge in my first dom.
But I am just curious on the kernel that the system used.
Why would it not use a sarge kernel instead of the current 2.6.18 one?

apt-get install kernel from the Sarge console shows lots of kernels there?
Or do you need to run the special xen kernels on the virtual boxes?

Also, for other newbies, I had one problem:
The vif error. Steve, could you not add a check to fix this when using the xen tools?

At any rate, to fix it, edit your /etc/xen/xend-config.sxp
Uncomment this line: (network-script network-bridge)
And comment this one: (network-script network-dummy)

Now stop xend and then start it again:
/etc/init.d/xend stop
/etc/init.d/xend start

Now try starting your newly created virtual machine.

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Fri 29 Jun 2007 at 12:24
[ View Steve's Scratchpad | View Weblogs ]

To answer you in brief:

Xen Kernels

You don't need to install a Kernel inside your guests, as they run the kernel from the host system. (The one that is referred to in the xen-tools.conf file.)

That means you can ignore the Sarge kernels.

vif defaults

Good idea, I will do that.

I can't force it, or update it, because people may have strange defaults but I can certainly warn about it.

Steve

[ Parent | Reply to this comment ]

Posted by lters (69.176.xx.xx) on Thu 12 Jul 2007 at 20:20
[ View lters's Scratchpad | View Weblogs ]
What if I wanted to run a new kernel on the virtual Debian?

Is that possible? Or how does that work?

It seems to be looking for kernels in /boot...

[ Parent | Reply to this comment ]

Posted by Steve (62.30.xx.xx) on Thu 12 Jul 2007 at 20:22
[ View Steve's Scratchpad | View Weblogs ]

Either:

  • Install a new kernel package upon the host machine.
  • Build your own Xen-kernel from source and install it somewhere.

Once you have a new kernel built you can specify it in the configuration file and it will be used by your guest.

Steve

[ Parent | Reply to this comment ]

Posted by AndrBrn (151.56.xx.xx) on Tue 14 Aug 2007 at 19:07
Nice article and nice scripts, even if I had to make some minor changes...
My dom0 system is Etch (all the packages come from the Etch repository) and to install both an Etch-based VM and a Sarge-based VM, setting the gdm role, I had to replace all the occurrences of "/etc/X11/gdm/gdm.conf" with "/etc/gdm/gdm.conf" in the gdm script in /etc/xen-tools/role.d folder and I had to change the last line (to make the VNC section the default) with this one:

perl -pi.bak -e 's/^0=Standard\n//g ; s/^\[servers\]/\[servers\]\n0=VNC/g' ${prefix}/etc/gdm/gdm.conf

It is a little bit dirty, but it works both for Sarge-based VMs where there is the "0=Standard" string in gdm.conf and for Etch-based VMs where the [servers] section in gdm.conf is empty, so there is no "0=Standard" string to replace.

And finally, to install gdm, VNC and other packages for a Sarge-baded VM, I had to replace the "stable" string with the "sarge" string in the sources.list file as soon as debootstrap created the root hierarchy (I changed the file while the xen-create-image script was running): is it a bug in the debootstrap package?

Andrea

[ Parent | Reply to this comment ]

Posted by Steve (80.68.xx.xx) on Fri 17 Aug 2007 at 10:44
[ View Steve's Scratchpad | View Weblogs ]

Thanks for the comment!

I think this would have been better to have been posted as a bug report, or directly to me though. I'll try to take a look at both issues over the weekend.

Steve

[ Parent | Reply to this comment ]

Posted by AndrBrn (151.71.xx.xx) on Fri 17 Aug 2007 at 11:57
Yes, I agree that the primary target for such things should be the Debian BTS, but I consider this site very helpful so I thought that posting a kind of bug-fixing here could be worthwhile.
But I will keep your hint for next time... ;-)

Andrea

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 828 votes ~ 10 comments )