Automating the creation of docker images

Posted by Steve on Sun 5 Jan 2014 at 22:57

In our previous introduction to docker we demonstrated how to create images, and manipulate them. That was useful but not as useful as the use of "Dockerfiles", which allow you to automatically build your own images.

In our previous article we used two images:

  • ubuntu
    • An image which was pulled from the internet.
  • debootstrap/wheezy

What we're going to do now is start from our debootstrap image and automate the creation of a system which will run ssh. The resulting image can be further extended in the future to run additonal services - and we'll demonstrate that by adding memcached to it, giving an image with both services running.

To get started save the following to the file Dockerfile.runit:

#
# Simple dockerfile for an ssh + server.
#
# Build it like so:
#
#   root@host~# docker build -t=steve/runit - < Dockerfile.runit
#
# Launch the generated image like so:
#
#   root@host~# docker run -d -p 2222:22 steve/runit
#
# Connect like so, with the root password being "mypasswd".
#
#   $ ssh -p 2222 root@localhost
#
# Steve
# --
#



#
#  From this base-image / starting-point
#
FROM debootstrap/wheezy

#
#  Authorship
#
MAINTAINER steve@steve.org.uk

#
# Update apt
#
RUN apt-get update -q -q
RUN apt-get upgrade --yes --force-yes

#
# Install utitiles
#
RUN apt-get install less sudo screen --yes --force-yes

#
# Install runit
#
RUN apt-get install runit --yes --force-yes

#
# Install SSH
#
RUN apt-get install openssh-server openssh-client --yes --force-yes

#
# Setup a root password; simple enough to remember, but hard enough that
# it won't be cracked immediately.  (ha!)
#
RUN echo "root:mypasswd" | chpasswd

#
# Expose the SSH port
#
EXPOSE 22

#
#  Now make sure that runit will launch SSHD, via runit.
#
#  NOTE: Remember runit will launch /etc/service/sshd/run
#
RUN mkdir /etc/service/sshd
RUN /bin/echo -e '#!/bin/sh' > /etc/service/sshd/run
RUN /bin/echo -e 'exec /usr/sbin/sshd -D' >> /etc/service/sshd/run

#
#  Make sure our run-script is executable.
#
RUN chown root.root /etc/service/sshd/run
RUN chmod 755 /etc/service/sshd/run


#
# Finally launch runit.
#
ENTRYPOINT ["/usr/sbin/runsvdir-start"]

Once you've done that you can create a new image, which will be called steve/runit by exectuing "docker build". The build-process reads the specified file, and executes the commands from it.

As you can infer from the example there are several things you can do in these Dockers - run commands, expose ports, etc. There is a complete list of valid commands listed on the docker website.

For the moment we'll gloss over the details and start the image-creation process:

root@shelob:~# docker build -t=steve/runit - < Dockerfile.runit
Uploading context 3.072 kB
Step 1 : FROM debootstrap/wheezy
 ---> 9b1b1f218cb1
Step 2 : MAINTAINER steve@steve.org.uk
 ---> Running in aa3f259416a5
 ---> 375bea46d0e5
Step 3 : RUN apt-get update -q -q
..
Step 15 : ENTRYPOINT ["/usr/sbin/runsvdir-start"]
 ---> Running in c3038f9ebb9f
 ---> 8e55370ca54a
Successfully built 8e55370ca54a

If all goes smoothly you'll find that when you run "docker images" you have a new image which is named steve/runit like so:

root@shelob:# docker images
REPOSITORY           TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
steve/runit          latest              8e55370ca54a        35 seconds ago      384.3 MB
debootstrap/wheezy   latest              9b1b1f218cb1        2 minutes ago       218.5 MB

Now we can launch it, forwarding port 1022 to the ssh deamon which will listen on port 22:

root@shelob:~# docker run -d -p 1122:22 steve/runit
827021b6c06cc0b508edf84a907d4b996d972e084471a39b7acbda25e2ed5a3d

Does it work? Let us find out:

root@shelob:~# ssh -p 1122 root@localhost
The authenticity of host '[localhost]:1122 ([::1]:1122)' can't be established.
ECDSA key fingerprint is a0:25:4f:e5:47:88:fe:c0:78:66:cb:a2:c1:cb:2f:a5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[localhost]:1122' (ECDSA) to the list of known hosts.
root@localhost's password:
Linux shelob 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64
...

NOTE: The root password is mypasswd, which was set in the Dockerfile.runit we fed to the build-process.

We've managed to achieve the creation of a new image, running ssh, in a roundabout fashion, and now I can reveal why we did that - The docker tool, as we say last time, will only allow a single process to be specified as the startup-command.

If we wanted a docker container to run both ssh and nginx, for example, we'd be screwed - we couldn't specify both daemons as the startup command and would instead have to choose one or the other.

To counter that we specified runsvdir-start as our startup-command, from the runit package - and this means we can link in services, via /etc/service, and cause more than one to launch easily.

NOTE: We recently posted a brief introduction to using runit.

As an example we might want to create a docker image which runs both SSH and memcached on startup. Because our dockerfile allows us to specify a starting image to build upon we can just add the new service, memcached, to the image we've just created - we don't need to start from scratch.

Save the following contents to the file Dockerfile.memcache:


#
#  From this base-image / starting-point
#
FROM steve/runit

#
#  Authorship
#
MAINTAINER steve@steve.org.uk

#
# Install memcached
#
RUN apt-get install memcached --yes --force-yes

#
# Ensure it listens externally
#
RUN sed -i "/127.0.0.1/d" /etc/memcached.conf

#
# Expose the memcached port
#
EXPOSE 11211

#
#  Now make sure that runit will launch memcached
#
#  NOTE: Remember runit will launch /etc/service/mem/run
#
RUN mkdir /etc/service/mem
RUN /bin/echo -e '#!/bin/sh' > /etc/service/mem/run
RUN /bin/echo -e 'exec //usr/bin/memcached -u root -m 128' >> /etc/service/mem/run

#
#  Make sure our run-script is executable.
#
RUN chown root.root /etc/service/mem/run
RUN chmod 755 /etc/service/mem/run


#
# Finally launch runit.
#
ENTRYPOINT ["/usr/sbin/runsvdir-start"]

Using this file we can now build a new image:

root@shelob:~# docker build -t=steve/memssh - < Dockerfile.memcache

The end result will be a system which runs both SSH and memcached, if we forward both sets of ports when we launch it:

root@shelob:~# docker run -d  -p 11211:11211 -p 1234:22 steve/memssh

Once you've launched it you'll see that port 11211 is forwarded to the memcached instance on the guest and port 1234 is forwarded to the SSH port. The end result is we've built a guest-image which is running two services, based on the ssh-only image we'd created previously.

This concludes our coverage of docker - but if you use it for interesting things your submissions are welcome.

 

 


Posted by Anonymous (84.58.xx.xx) on Tue 7 Jan 2014 at 20:40
wow - what a productive january, Steve!
Thanks a lot for your recent articles.

best wishes
/thorsten

[ Parent | Reply to this comment ]

Posted by Steve (90.220.xx.xx) on Wed 8 Jan 2014 at 16:29
[ View Steve's Scratchpad | View Weblogs ]

I only hope they're useful - I think I rushed through the coverage of runit a little too quickly, and didn't give enough useful examples, but I wanted to get the minimum required stuff covered before going back to docker.

Docker is proving to be pretty interesting to me right now, not because it does anything novel or new, but because of the simpliicty and the documentation inherant in writing the recipes.

Steve

[ Parent | Reply to this comment ]

Posted by Anonymous (157.159.xx.xx) on Thu 6 Mar 2014 at 13:57
I have tested this and seem to have noticed some errors reported doing a ps, when the container with the sshd is started, and indeed, can't connect with ssh to it :-/

How can we try and debug the container's startup if things go wrong ?

[ Parent | Reply to this comment ]

Posted by Steve (176.27.xx.xx) on Thu 6 Mar 2014 at 14:15
[ View Steve's Scratchpad | View Weblogs ]

Generally I comment out the last-line of the dockerfile, and rebuild the image (which will be fast).

Once you've done that you can launch the image with bash as the command, rather than the hardwired startup-command. e.g. The docker file discussed here could be launched like so:

docker run -t -i -p 2222:22 steve/runit /bin/bash

This would start it with a terminal, and interactively.

Now you can run the start-up command, in our example "/usr/sbin/runsvdir-start", and see what the service state is by looking at "sv status /etc/service/*", and take it from there.

Steve

[ Parent | Reply to this comment ]

Posted by olberger (157.159.xx.xx) on Thu 6 Mar 2014 at 14:49
Actually, the issue was that openssh-server does need /var/run/sshd which isn't created by the startup script.

I think the following line is needed before the one starting sshd :

RUN /bin/echo -e 'if [ ! -d /var/run/sshd ]; then mkdir -p /var/run/sshd ; fi' >> /etc/service/sshd/run

[ Parent | Reply to this comment ]

Posted by Steve (176.27.xx.xx) on Thu 6 Mar 2014 at 17:17
[ View Steve's Scratchpad | View Weblogs ]

Good catch - That is already present in my Dockerfile, from the small collection I keep.

Steve

[ Parent | Reply to this comment ]

Posted by olberger (157.159.xx.xx) on Fri 7 Mar 2014 at 10:39
Well, it is possible to connect to the container with lxc-attach and see what happens inside it. See http://stackoverflow.com/questions/20932357/docker-enter-running- container-with-new-tty for details

[ Parent | Reply to this comment ]

Posted by olberger (157.159.xx.xx) on Thu 6 Mar 2014 at 17:00
The Dockerfiles may include the following :

ENV DEBIAN_FRONTEND noninteractive
ENV DEBIAN_PRIORITY critical
ENV DEBCONF_NOWARNINGS yes

which may help get rid of some error messages... while not necessarily suitable for all images...

[ Parent | Reply to this comment ]

Posted by olberger (82.238.xx.xx) on Thu 1 May 2014 at 15:15
It seems that runit-run used to provide a way to use runit in the classical init.d way. But it has apparently been removed from Debian (still in Ubuntu, it seems).

In any case, at the moment, the Debian base image created by mkimage-debootstrap.sh will add a policy-rc.d which disables invoke-rc.d invocations.

I don't feel satisfied with this as it prevents existing restart invocations to perform, for instance for stopping a postgresql server while backing up, and likes.

Have you investigated this issue, and maybe have ideas on the use of runit-run or modifying policy-rc.d to divert some invocations of invoke-rc.d and do the equivalent sv commands ?

[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 123 votes ~ 0 comments )