Difference between revisions of "Backup with duplicity"
m (→Remote backup with duplicity (Tomcat - PostgreSQL)) |
m (→Remote backup with duplicity (Tomcat - PostgreSQL)) |
||
Line 179: | Line 179: | ||
BACKUP_DIR="ftp://user@ftp.domain.es/backup" | BACKUP_DIR="ftp://user@ftp.domain.es/backup" | ||
# BACKUP_DIR="sftp://user@sftp.domain.es//path/to/backup" | # BACKUP_DIR="sftp://user@sftp.domain.es//path/to/backup" | ||
− | |||
export FTP_PASSWORD="WhateverPasswordYouSetUp" | export FTP_PASSWORD="WhateverPasswordYouSetUp" | ||
## END CONFIG ## | ## END CONFIG ## |
Revision as of 10:16, 12 June 2013
duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.
Duplicity can be installed in Debian / Ubuntu as simple as:
$ sudo aptitude install duplicity
But it not in the CentOS / RedHat default repositories, so you need to install from another source. This script will help in this installation process:
#!/bin/bash
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/duplicity-0.6.18-1.el6.x86_64.rpm
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/ncftp-3.2.4-1.el6.x86_64.rpm
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/librsync-0.9.7-15.el6.x86_64.rpm
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/python-GnuPGInterface-0.3.2-6.el6.noarch.rpm
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/python-boto-2.5.2-1.el6.noarch.rpm
rpm -Uvh duplicity-0.6.18-1.el6.x86_64.rpm ncftp-3.2.4-1.el6.x86_64.rpm librsync-0.9.7-15.el6.x86_64.rpm python-GnuPGInterface-0.3.2-6.el6.noarch.rpm python-boto-2.5.2-1.el6.noarch.rpm
See also:
- Duplicity Backup Howto
- Duplicity URL Format
- Installation and configuration of duplicity for encrypted SFTP remote backup
- Playing with Duplicity backup and restore tool and Amazon S3
Problems deleting old backups
You have configured Duplicity to make a full backups every 7 days, and deleting backups older than 14 days to save space. What happen if you see a message like this?
Tue May 1 00:00:00 2012 Wed May 2 00:00:00 2012 Which can't be deleted because newer sets depend on them
Duplicity uses rsync, which contains incremental changes. Those files won't be deleted because, even though the backup maybe older than 7 days, there are backups which are incrementals and younger than 7 days.
So, after 2 weeks have passed, those files will be deleted, since the full backup and the incremental backups are now 14 days old, and there exists a full backup newer than the full and incremental backups.
Remote backup with duplicity (JBoss)
#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
MYSQL_PASS=""
OPENKM_DB="okmdb"
OPENKM_HOME="/home/openkm"
JBOSS_HOME="$OPENKM_HOME/jboss-4.2.3.GA"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DIR="ftp://user@ftp.domain.es/backup"
export FTP_PASSWORD="WhateverPasswordYouSetUp"
## END CONFIG ##
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
# Stop JBoss
/etc/init.d/jboss stop
# Clean logs
echo "Clean JBoss temporal files."
# rm -rf $JBOSS_HOME/server/default/log
rm -rf $JBOSS_HOME/server/default/tmp
rm -rf $JBOSS_HOME/server/default/work
rm -rf $JBOSS_HOME/server/default/data/wsdl
# Backup de MySQL
if [ -n "$MYSQL_PASS" ]; then
echo "* Backuping MySQL data from $OPENKM_DB...";
mysqldump -h localhost -u root -p$MYSQL_PASS $OPENKM_DB > $DATABASE_EXP/mysql_$OPENKM_DB.sql
echo "-------------------------------------";
fi
# Backup and purge old backups
duplicity remove-older-than 3M --force $BACKUP_DIR/$HOST
if [ $(date +%u) -eq 7 ]; then
echo "*** Full Backup ***"
duplicity full --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
else
echo "*** Incremental Backup ***"
duplicity --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
fi
# Start JBoss
/etc/init.d/jboss start
echo -e "\n### END: $(date +"%x %X") ###"
# Status
echo "=================================";
duplicity collection-status $BACKUP_DIR/$HOST
unset FTP_PASSWORD
echo "=================================";
Remote backup with duplicity (Tomcat - MySQL)
#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
MYSQL_PASS=""
OPENKM_DB="okmdb"
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.27"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DIR="ftp://user@ftp.domain.es/backup"
# BACKUP_DIR="sftp://user@sftp.domain.es//path/to/backup"
# BACKUP_DIR="smb://windowscomputer/harddrive/folder"
export FTP_PASSWORD="WhateverPasswordYouSetUp"
## END CONFIG ##
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
# Stop Tomcat
/etc/init.d/tomcat stop
# Clean logs
#echo "Clean Tomcat temporal files."
#rm -rf $TOMCAT_HOME/logs/*
#rm -rf $TOMCAT_HOME/temp/*
#rm -rf $TOMCAT_HOME/work/Catalina/localhost
# Backup de MySQL
if [ -n "$MYSQL_PASS" ]; then
echo "* Backuping MySQL data from $OPENKM_DB...";
mysqldump -h localhost -u root -p$MYSQL_PASS $OPENKM_DB > $DATABASE_EXP/mysql_$OPENKM_DB.sql
echo "-------------------------------------";
fi
# Backup and purge old backups
duplicity remove-older-than 1M --force $BACKUP_DIR/$HOST
if [ $(date +%u) -eq 7 ]; then
echo "*** Full Backup ***"
duplicity full --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
RETVAL=$?
else
echo "*** Incremental Backup ***"
duplicity --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
RETVAL=$?
fi
[ $RETVAL -eq 0 ] && echo "*** SUCCESS ***"
[ $RETVAL -ne 0 ] && echo "*** FAILURE ***"
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
# Status
echo "=================================";
duplicity collection-status $BACKUP_DIR/$HOST
unset FTP_PASSWORD
echo "=================================";
Remote backup with duplicity (Tomcat - PostgreSQL)
#!/bin/bash
#
## BEGIN CONFIG ##
HOST=$(uname -n)
OPENKM_DB="okmdb"
OPENKM_HOME="/home/openkm"
TOMCAT_HOME="$OPENKM_HOME/tomcat-7.0.27"
DATABASE_EXP="$OPENKM_HOME/db"
BACKUP_DIR="ftp://user@ftp.domain.es/backup"
# BACKUP_DIR="sftp://user@sftp.domain.es//path/to/backup"
export FTP_PASSWORD="WhateverPasswordYouSetUp"
## END CONFIG ##
if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
echo -e "### BEGIN: $(date +"%x %X") ###\n"
rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP
# Stop Tomcat
/etc/init.d/tomcat stop
# Clean logs
#echo "Clean Tomcat temporal files."
#rm -rf $TOMCAT_HOME/logs/*
#rm -rf $TOMCAT_HOME/temp/*
#rm -rf $TOMCAT_HOME/work/Catalina/localhost
# Backup de PostgreSQL
echo "* Backuping PostgreSQL data from $OPENKM_DB..."
su postgres -c "pg_dump -Fc -b $OPENKM_DB" > $DATABASE_EXP/pg_$OPENKM_DB.dmp
# Databases optimizations
su postgres -c "vacuumdb -a -z" > /dev/null
su postgres -c "reindexdb -a -q" 2> /dev/null
# Backup and purge old backups
duplicity remove-older-than 1M --force $BACKUP_DIR/$HOST
if [ $(date +%u) -eq 7 ]; then
echo "*** Full Backup ***"
duplicity full --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
RETVAL=$?
else
echo "*** Incremental Backup ***"
duplicity --no-encryption $OPENKM_HOME $BACKUP_DIR/$HOST
RETVAL=$?
fi
[ $RETVAL -eq 0 ] && echo "*** SUCCESS ***"
[ $RETVAL -ne 0 ] && echo "*** FAILURE ***"
# Start Tomcat
/etc/init.d/tomcat start
echo -e "\n### END: $(date +"%x %X") ###"
# Status
echo "=================================";
duplicity collection-status $BACKUP_DIR/$HOST
unset FTP_PASSWORD
echo "=================================";