Securing distcc with chroot and OpenVPN
Posted by thomas on Mon 6 Jun 2005 at 23:13
This is a short guide on how to set up distcc to run in chroot and send/receive traffic over an encrypted tunnel using the OpenVPN software.
We've mentioned speeding up compilation with distcc previously - if you've never used it before that will serve as a good introduction to the tool.
The OpenVPN software is a versatile VPN solution which allows anything from joining networks as a VPN to allowing road-warriors to connect to a system remotely.
There are several benefits of this approach:
- Both distcc and OpenVPN runs as unprivilaged users in a chroot, which restricts the commands available to the client
- distcc accepts jobs only from clients that has a certificate issued by the OpenVPN server
- distcc listens for job on a private address space (10.8.0.0/24)
- There is no need to create additional user accounts on the server, which you had to do if you were using SSH
Of course there are some drawbacks too:
- Since the connections are made over OpenVPN it's slower.
- You need to manually update the chroot when a new version of distccd or gcc is available, this is however made easier with the available script.
- You need to issue certificates and distribute them to the client(s).
Setting up distccSince openvpn.net/howto.html#pki covers all the steps in a detailed manner, we're only going to touch this subject briefly.
To install openvpn use:
apt-get install openvpnCopy the /usr/share/doc/openvpn/examples/easy-rsa directory to /etc/openvpn and change its permissions:
cd /etc/openvpn cp -R /usr/share/doc/openvpn/examples/easy-rsa/ ./ chmod 0700 easy-rsa/Edit /etc/openvpn/easy-rsa/vars, and uncompress the openssl.cnf.gz file by running :
cd /etc/openvpn/easy-rsa gunzip openssl.cnf.gzCreate the CA key:
. ./vars ./clean-all ./build-caGenerate a certificate and private key for the server:
./build-key-server serverGenerate client certificate(s):
./build-key client1(Where "client1" is the identifier for the client - Always use a unique common name for each client.)Diffie Hellman parameters must be generated for the OpenVPN server. This can be achieved by running:
./build-dhCopy the necessary keys to /etc/openvpn:
cp easy-rsa/keys/{ca.crt,server.crt,server.key,dh1024.pem} ./Now you need to adjust the server configuration:
tls-server port 1194 proto udp dev tun ca ca.crt cert server.crt key server.key dh dh1024.pem server 10.8.0.0 255.255.255.0 ifconfig-pool-persist ipp.txt keepalive 10 120 comp-lzo chroot /etc/openvpn user nobody group nogroup persist-key persist-tun status openvpn-status.log log openvpn.log verb 3 mute 20The client configuration should now be very straightforward. For each client you'll need to setup something similar to:
dev tun proto udp remote distcc.example.com ns-cert-type server client ca ca.crt cert client1.crt key client1.key chroot /etc/openvpn user nobody group nogroup comp-lzo persist-tun persist-key status openvpn-status.log log openvpn.log verb 3 mute 20
Install distcc if you've not installed it previously:
apt-get install distccChange the "ALLOWEDNETS" setting in /etc/default/distcc to read:
ALLOWEDNETS="127.0.0.1 10.8.0.0/24"(This allows the connections to be accepted from our VPN network 10.8.0.0/24).
Run the chroot script available at www.northernsecurity.net/download/distccd-chroot and copy the init.d script (www.northernsecurity.net/download/distccd-chroot.init) file to /etc/init.d/distcc.
(These files are both mirrored here)
If everything went well /var/chroot/distccd/ should be created containing all necessary files.
Start distccd with "/etc/init.d/distcc start" and verify that it runs in the chroot.
# ps -fe|grep distccd|awk '{print $2}' 29302 # ls -l /proc/29302/root lrwxrwxrwx 1 distccd nogroup 0 2005-06-01 01:41 /proc/22628/root -> /var/chroot/distccdFrom the the distccd logfile you should see something to indicate it is successfully listening upon the private network:
distccd[29301] (dcc_listen_by_addr) listening on 10.8.0.1:3632The client should then be able to run distcc just like normal, except that the server is located at 10.8.0.1.
$ export DISTCC_VERBOSE=1 # We want details $ export DISTCC_HOSTS='10.8.0.1'Watching with jobs with distccmon-text you should see connection attempts as follows:
18929 Preprocess util_ebcdic.c 10.8.0.1[0] 11311 Compile util_time.c 10.8.0.1[0]Log on the client and if things work correctly you'll see something like this:
distcc[191] exec on 10.8.0.1: cc -g -O2 -pthread -c modules.c -o modules.o distcc[191] (dcc_note_state) note state 2, file "modules.c", host "10.8.0.1" distcc[191] (dcc_connect_by_name) connecting to 10.8.0.1 port 3632 distcc[191] (dcc_connect_by_addr) started connecting to 10.8.0.1:3632
