Debian Administration

That's very interesting and helpful, I'm looking forward to more parts.

A couple of minor points:

1. We should be using aptitude not apt-get these days. It's a small point, but for most purposes aptitude is a drop in replacement and it does offer genuine advantages over apt-get

2. You should not edit the sudoers file directly with an editor or echoing things to it. The sudo man page makes it quite clear you should use the included visudo tool. I know it's something we all do, but we shouldn't propagate it.

--
"It's Not Magic, It's Work"
Adam

The biggest advantage of BackupPC is its pooling mechanism. If you have the same file across several systems, only one copy is stored on the backup machine (via hard-links in the pool). Future backups will not store another copy of each file, but rather link to the pool. If you have several like systems this is a very space efficient method for backups as the pool is common to all clients.

Also worth mentioning is that general BackupPC tuning is to store your pool on reiserfs or xfs, mounted with noatime as you will get significant improvements. Other tips here: http://backuppc.wiki.sourceforge.net/speedupbackups

Mooi verhaal, Brum!

gr. DB

Hi ! Thanks for the article !

I have to insist on major BackupPC features, which makes it unique:
- BackupPC is disk based backup and not tape based.
- It can use smb, ssh, rsync, etc to backup clients, so no client agent has to be installed.
- It has a nice web interface.
- It has a clever pooling scheme which minimizes disk storage and disk I/O.
Identical files across multiple backups of the same or different PC are
stored only once resulting in substantial savings in disk storage and disk
writes. Also known as "data deduplication".

This "data deduplication" is very interesting and only possible on disk based backups: If you backup 200 Windows PCs, only a single copy of the system will be on the backup disk (e.g. not 2x200GB=400GB but only 2GB + compression). It's as if BackupPC only stored the differences, so that you can also have a very long backup retention policy without needing lots of disks.

A must try !

Ludovic (BackupPC Debian package maintainer :-)

Do you not also need to change the "RsyncClientRestoreCmd" setting too?

hello
I use backuppc for about 1.5 year, and i.m happy witch it.

Why use backuppc and sudo, u can easy use root acount to ssh.
If you have acces to run rsync witch id=0, you can simply overwrite any file, so u can change /etc/passwd.

My idea of use backuppc was to create ssh tunell to rsync demon, this idea have some benefits:
- nobady from backuppc can log to backupded host
- nobady from backuped host cant log to backuppc host
- NAT is no more a problem, connection direction is from backuped host to backuppc.

Hire you have my help script, to create cut/paste:

cat create_backuppc_paste.sh
#!/bin/bash

# free port to forward ssh connectin
# each host has own port
PORT=6054
# rsync speed ( 128 = 1Mb/s )
SPEED=512

# backuped host
REMOTE_HOST_NAME="my_hostname"

# rsuncd password (from generator)
RSYND_PASSWORD="Mu6Kacxx"

# command to do ssh on backuppp
SSH_NA_BACKUPPC="ssh backuppc"


###
# Rsync config
###
#in debian
RSYNC_LOCATION="/usr/bin/rsync"
# in OpenBSD
# RSYNC_LOCATION="/usr/local/bin/rsync"

BACKUP_DIR="/"
# exclude from backup
# if you use LVN snapshot put /backup/ prefix
EXCLUDE="/backup/dev /backup/vz/sharedvz/2027/root/old_nagios /backup/vz/sharedvz/2027/root/dev "

###
# LVN SNAPSHOT
###

# YES OR NO
USE_LVN_SNAPSHOT="YES"

# VOLUME GROUP
VG="/dev/main"
# LOGICAL VOLUME
LV="root"

# default mount point = /mnt/rootmirror
if [[ "$USE_LVN_SNAPSHOT" == "YES" ]] ; then BACKUP_DIR="/mnt/rootmirror$BACKUP_DIR" ; fi

palza(){
echo ;echo;echo
echo "# press enter to continue ... "
read
clear
}


clear
echo "#####################################################"
echo "# paste this to backuppc "
echo "#####################################################"
echo
echo "##################"
echo "# create tunel usre "
echo "#################"
echo "useradd -m $REMOTE_HOST_NAME -s /bin/true"
echo "# create ssh key for tunel user "
echo "su $REMOTE_HOST_NAME -s /bin/bash -c \"mkdir /home/$REMOTE_HOST_NAME/.ssh\" "
echo "su $REMOTE_HOST_NAME -s /bin/bash -c \"ssh-keygen -N '' -f /home/$REMOTE_HOST_NAME/.ssh/id_rsa\" "
echo "su $REMOTE_HOST_NAME -s /bin/bash -c \"cp /home/$REMOTE_HOST_NAME/.ssh/id_rsa.pub /home/$REMOTE_HOST_NAME/.ssh/authorized_keys\" "
palza

echo "##################"
echo "# create backuppc konfiguration "
echo "#################"

echo "cat >> /etc/backuppc/hosts << _EOF_"
echo "lh-$PORT-$REMOTE_HOST_NAME 0 backuppc
_EOF_
"

echo "cat >> /etc/backuppc/lh-$PORT-$REMOTE_HOST_NAME.pl << _EOF_"
echo "
#
# Rsync Backup (with rsync daemon on the client)
#

\\\$Conf{XferMethod} = 'rsyncd';

\\\$Conf{RsyncShareName} = ['backup'];

# if needed set a user name and password to access the remote shares
\\\$Conf{RsyncdUserName} = 'backuppc';
\\\$Conf{RsyncdPasswd} = '$RSYND_PASSWORD';
\\\$Conf{RsyncdClientPort} = $PORT;
_EOF_
"

echo "cp /etc/hosts /etc/hosts-`date -I`.backup"
echo "sed -i -e \"s/localhost/localhost lh-$PORT-$REMOTE_HOST_NAME/\" /etc/hosts"

echo "# reload backuppc "
echo "/etc/init.d/backuppc reload"
palza


echo "#####################################################"
echo "# paste this to remote host $REMOTE_HOST_NAME "
echo "#####################################################"
echo
echo "##################"
echo "# create a private ssh key on $REMOTE_HOST_NAME"
echo "#################"
echo "mkdir -p /root/.ssh"
echo "cat > /root/.ssh/id_rsa_backuppc << _EOF_"
$SSH_NA_BACKUPPC "cat /home/$REMOTE_HOST_NAME/.ssh/id_rsa"
echo "_EOF_"
echo "chmod og-rw /root/.ssh/id_rsa_backuppc"
echo
palza


echo "##################"
echo "# create and run tunel on $REMOTE_HOST_NAME"
echo "#################"
SCRYPT="/root/bin/redirect_port_from_backuppc.sh"
echo "mkdir -p /root/bin"
echo "cat > $SCRYPT << _EOF_ "
echo "#!/usr/bin/env bash
exec &> >(while read line; do logger -t \"\\\$0\" -i \"\\\$line\"; done)

while true
do
/usr/bin/ssh -NR 127.0.0.1:$PORT:127.0.0.1:873 -i /root/.ssh/id_rsa_backuppc -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=300 -p 6667 -l $REMOTE_HOST_NAME 217.17.45.189
sleep 60
done
"
echo "_EOF_"
echo "chmod a+x $SCRYPT"
echo "####"
echo "# tunell start "
echo "####"
echo "# to crontab "
echo "( crontab -l ; echo \"#Start tunell ssh to backuppc \"; echo \"@reboot /root/bin/redirect_port_from_backuppc.sh &\" ) | crontab -"
echo "# first run: "
echo "nohup /root/bin/redirect_port_from_backuppc.sh &"
palza


echo "##################"
echo "# rsync konfiguration on $REMOTE_HOST_NAME "
echo "#################"
PRE_BACKUP="/root/bin/pre_backup.sh"
POST_BACKUP="/root/bin/post_backup.sh"
echo "cat > /etc/rsyncd.conf << _EOF_"
echo "timeout = 600
[backup]
path = $BACKUP_DIR
uid = 0
gid = 0
max connections = 1
comment = backap
auth users = backuppc
secrets file = /etc/rsyncd.secrets
exclude = $EXCLUDE
read only = yes
pre-xfer exec = $PRE_BACKUP
post-xfer exec = $POST_BACKUP
_EOF_
"
echo "cat > /etc/rsyncd.secrets << _EOF_"
echo "backuppc:$RSYND_PASSWORD
_EOF_
"
echo "chmod og-rw /etc/rsyncd.secrets "
palza


echo "##################"
echo "# pre and post backup on $REMOTE_HOST_NAME "
echo "#################"
echo "####"
echo "# file $PRE_BACKUP"
echo "####"
echo "cat > $PRE_BACKUP << _EOF_"
echo "#!/usr/bin/env bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin :/usr/bin/X11
exec &> >(while read line; do logger -t \"\\\$0\" -i \"\\\$line\"; done)
set -x
"

if [ "$USE_LVN_SNAPSHOT" == "YES" ] ;then
echo "
VG='$VG'
LV='$LV'
MIRROR='rootmirror'

mkdir -p /mnt/\\\$MIRROR/backup

lvcreate -n \\\$MIRROR -s -L 3G \\\$VG/\\\$LV
mount \\\$VG/\\\$MIRROR /mnt/\\\$MIRROR/backup

cp -a /boot/* /mnt/\\\$MIRROR/backup/boot/
"
fi
echo "_EOF_

"
echo "chmod a+x $PRE_BACKUP"

echo "####"
echo "# file $POST_BACKUP "
echo "####"
echo "cat > $POST_BACKUP << _EOF_"
echo "#!/usr/bin/env bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin :/usr/bin/X11
exec &> >(while read line; do logger -t \"\\\$0\" -i \"\\\$line\"; done)
set -x
"

if [ "$USE_LVN_SNAPSHOT" == "YES" ] ;then
echo "
VG='$VG'
LV='$LV'
MIRROR='rootmirror'

umount /mnt/\\\$MIRROR/backup
lvremove -f \\\$VG/\\\$MIRROR
"
fi
echo "
_EOF_

"

echo "chmod a+x $POST_BACKUP"

if [ "$USE_LVN_SNAPSHOT" == "YES" ] ;then
echo "mkdir -p /mnt/rootmirror"
fi
palza

echo "####"
echo "# start rsyncd on $REMOTE_HOST_NAME "
echo "####"
RSYNC="$RSYNC_LOCATION --daemon --config /etc/rsyncd.conf --bwlimit=$SPEED --address=127.0.0.1"
echo "#to cronab"
echo "( crontab -l ; echo \"#Odpalenie rsyncd przys starcie \"; echo \"@reboot $RSYNC \" ) | crontab -"
echo "# start rsyncd "
echo "$RSYNC"
palza

echo "####"
echo "# rsync test on backuppc - list all files that will be backuped. "
echo "####"
echo "RSYNC_PASSWORD=$RSYND_PASSWORD rsync -a -v -n --port=$PORT backuppc@127.0.0.1::backup /tmp/jakistest"





Wolodyj

I've followed this guide, and it's working great.

But I have one question.

I presumed that an incremental backup (via rsync) would only copy over the changed files. However it seems to copy over all files, then only store the changed ones.

There is one machine we want to backup which has to be done via the net, so copying all files nightly takes a large chunk of our bandwidth.

Is there a solution to this, as I'd love to keep using this program rather than going back to my custom rsync script.

Coops.

Still having problems with backuppc:/

Thanks for a nice article.

If you are looking for a backup solution then check out dar. http://dar.sourceforge.net/