Difference between revisions of "Backup scripts"

Latest revision as of 15:01, 10 March 2014

These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/. Also recommend the article DAR differential backup mini-howto.

To install the cron job, run:

$ sudo crontab -e

And add these lines according to your personal configuration:

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee -a /root/backup.log

Or, if you want to separate log reports by date:

MAILTO=nomail@openkm.com
@weekly /root/backup.sh | tee /root/backup.$(date +\%Y.\%m.\%d_\%H.\%M.\%S).log

MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.

This script will export all databases from a MySQL installation:

#!/bin/bash
#
## BEGIN CONFIG ##
MYSQL_PASS=""
DATABASE_EXP="/home/openkm/db"
## END CONFIG ##

rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP

# Backup de MySQL
if [ -n "$MYSQL_PASS" ]; then
  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $2 }');
 
  for DB in $MYSQL_DBS; do
    if [[ $DB != "mysql" && $DB != "test" && $DB != "information_schema" && $DB != "performance_schema" ]]; then
      echo "* Backuping MySQL data from $DB..."
      mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
    fi
  done
fi

This script will export all databases from a PostgreSQL installation:

#!/bin/bash
#
## BEGIN CONFIG ##
DATABASE_EXP="/home/openkm/db"
## END CONFIG ##

rm -rf $DATABASE_EXP
mkdir -p $DATABASE_EXP

# Backup de PostgreSQL
POSTGRESQL_DBS=$(su postgres -c "psql -l" | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $1 }');

for DB in $POSTGRESQL_DBS ; do
  if [[ $DB != "template0" && $DB != "template1" && $DB != "postgres" ]]; then
    echo "* Backuping PostgreSQL data from $DB..."
    su postgres -c "pg_dump -Fc -b $DB" > $DATABASE_EXP/pg_$DB.dmp
  fi
done

For more information take a look at Crontab quick reference

Difference between revisions of "Backup scripts"

Latest revision as of 15:01, 10 March 2014

Navigation menu

Views

Personal tools

Navigation

Search

Tools

@@ Line 1: / Line 1: @@
-These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/.
+{{TOCright}} __TOC__
+These backup scripts use rsync to minimize network load and creates incremental backups, preserving last four backups. For more info, read http://www.mikerubel.org/computers/rsync_snapshots/. Also recommend the article [http://dar.linux.free.fr/doc/mini-howto/dar-differential-backup-mini-howto.es.html DAR differential backup mini-howto].
 To install the cron job, run:
@@ Line 11: / Line 13: @@
 @weekly /root/backup.sh | tee -a /root/backup.log
 </source>
-[http://adminschoice.com/crontab-quick-reference link Crontab quick reference]
 Or, if you want to separate log reports by date:
@@ Line 21: / Line 21: @@
 </source>
-== Remote backup with rsync ==
+{{Note|MAILTO may also be used to direct mail to multiple recipients by separating recipient users with a comma.}}
-<source lang="bash">
-#!/bin/bash
-#
-## BEGIN CONFIG ##
-HOST=$(uname -n)
-FILES="/home/openkm"
-BACKUP_DIR="/mnt/backup"
-## END CONFIG ##
-echo -e "### BEGIN: $(date +"%x %X") ###\n"
-# Stop JBoss
-/etc/init.d/jboss stop
-while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
-  sleep 5; echo ".";
-done
-# Copy to backup server
-rsync -apzhR --stats --delete --exclude=*~ --delete-excluded $FILES backup@server:$BACKUP_DIR/$HOST
-# Start JBoss
+This script will export all databases from a MySQL installation:
-/etc/init.d/jboss start
-echo -e "\n### END: $(date +"%x %X") ###"
-</source>
-== Remote backup with rsync and rotation ==
 <source lang="bash">
 #!/bin/bash
 #
 ## BEGIN CONFIG ##
-HOST=$(uname -n)
+MYSQL_PASS=""
-FILES="/home/openkm"
+DATABASE_EXP="/home/openkm/db"
-BACKUP_DIR="/mnt/backup"
 ## END CONFIG ##
-echo -e "### BEGIN: $(date +"%x %X") ###\n"
-# Stop JBoss
+rm -rf $DATABASE_EXP
-/etc/init.d/jboss stop
+mkdir -p $DATABASE_EXP
-# Copy to backup server
+# Backup de MySQL
-ssh backup@server "cd $HOST; rm -rf backup.3; mv backup.2 backup.3; mv backup.1 backup.2; mv backup.0 backup.1"
+if [ -n "$MYSQL_PASS" ]; then
-rsync -apzhR --stats --delete --exclude=*~ --delete-excluded --link-dest="$BACKUP_DIR/$HOST/backup.1" \
+  MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $2 }');
-$FILES backup@server:$BACKUP_DIR/$HOST/backup.0
+  for DB in $MYSQL_DBS; do
-# Start JBoss
+    if [[ $DB != "mysql" && $DB != "test" && $DB != "information_schema" && $DB != "performance_schema" ]]; then
-/etc/init.d/jboss start
+      echo "* Backuping MySQL data from $DB..."
-echo -e "\n### END: $(date +"%x %X") ###"
+      mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
+    fi
+  done
+fi
 </source>
-== Local backup with rsync and rotation ==
+This script will export all databases from a PostgreSQL installation:
-<source lang="bash">
-#!/bin/bash
-#
-## BEGIN CONFIG ##
-HOST=$(uname -n)
-FILES="/home/openkm"
-BACKUP_DIR="/mnt/backup"
-## END CONFIG ##
-echo -e "### BEGIN: $(date +"%x %X") ###\n"
-# Stop JBoss
-/etc/init.d/jboss stop
-# Calculate snapshot
-LAST_SNAPSHOT=`ls -ltr $BACKUP_DIR | tail -1 | awk {'print $8'} | cut -d . -f 2`
-NEW_SNAPSHOT=$((LAST_SNAPSHOT+1))
-# Copy to backup server
-rsync -apzhR --stats --delete --exclude=*~ --exclude="$JBOSS_HOME/cache" --delete-excluded \
---link-dest="$BACKUP_DIR/$HOST/backup.$LAST_SNAPSHOT" $FILES "$BACKUP_DIR/$HOST/backup.$NEW_SNAPSHOT"
-# Start JBoss
-/etc/init.d/jboss start
-echo -e "\n### END: $(date +"%x %X") ###"
-# Status
-echo "=================================";
-du -hs $BACKUP_DIR
-echo "*********************************";
-du -hs --time $BACKUP_DIR/*
-echo "=================================";
-df -h | grep "$BACKUP_DIR"
-echo "=================================";
-</source>
-== Remote backup with rdiff-backup ==
 <source lang="bash">
 #!/bin/bash
 #
 ## BEGIN CONFIG ##
-HOST=$(uname -n)
+DATABASE_EXP="/home/openkm/db"
-FILES="/home/openkm"
-BACKUP_DIR="/mnt/backup"
 ## END CONFIG ##
-echo -e "### BEGIN: $(date +"%x %X") ###\n"
-# Stop JBoss
-/etc/init.d/jboss stop
-# Backup and purge old backups
-rdiff-backup --remove-older-than 30B backup@server::$BACKUP_DIR/$HOST
-rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
-  --include $FILES --exclude '**' / backup@server::$BACKUP_DIR/$HOST
-# Start JBoss
-/etc/init.d/jboss start
-echo -e "\n### END: $(date +"%x %X") ###"
-# Status
-echo "=================================";
-rdiff-backup --list-increment-sizes backup@server::$BACKUP_DIR/$HOST
-echo "=================================";
-</source>
-More info about rdiff-backup:
-* [http://rdiff-backup.nongnu.org/ rdiff-backup home page]
-* [http://wiki.rdiff-backup.org/wiki/index.php/Main_Page rdiff-backup wiki]
-== Local backup to USB disk ==
-USB disk mount point can be defined in ''/etc/fstab'' as:
-  /dev/sdb1       /mnt/backup     ext4    defaults        0       0
-<source lang="bash">
-#!/bin/bash
-#
-## BEGIN CONFIG ##
-HOST=$(uname -n)
-MYSQL_PASS=""
-OPENKM_HOME="/home/openkm"
-JBOSS_HOME="$OPENKM_HOME/jboss-4.2.3.GA"
-DATABASE_EXP="$OPENKM_HOME/db"
-BACKUP_DIR="/mnt/backup"
-## END CONFIG ##
-if [ $(id -u) != 0 ]; then echo "You should run this script as root"; exit; fi
-echo -e "### BEGIN: $(date +"%x %X") ###\n"
 rm -rf $DATABASE_EXP
 mkdir -p $DATABASE_EXP
-# Mount disk
+# Backup de PostgreSQL
-if mount | grep "$BACKUP_DIR type" > /dev/null; then
+POSTGRESQL_DBS=$(su postgres -c "psql -l" | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) { print $1 }');
-  echo "$BACKUP_DIR already mounted";
-else
-  mount $BACKUP_DIR;
-fi
-# Stop JBoss
+for DB in $POSTGRESQL_DBS ; do
-/etc/init.d/jboss stop
+   if [[ $DB != "template0" && $DB != "template1" && $DB != "postgres" ]]; then
+     echo "* Backuping PostgreSQL data from $DB..."
-# Clean logs
+     su postgres -c "pg_dump -Fc -b $DB" > $DATABASE_EXP/pg_$DB.dmp
-echo "Clean JBoss temporal files."
+   fi
-rm -rf $JBOSS_HOME/server/default/log
+done
-rm -rf $JBOSS_HOME/server/default/tmp
-rm -rf $JBOSS_HOME/server/default/work
-# Backup de MySQL
-if [ -n "$MYSQL_PASS" ]; then
-   MYSQL_DBS=$(mysqlshow -h localhost -u root -p$MYSQL_PASS | awk '(NR > 2) && (/[a-zA-Z0-9]+[ ]+[|]/) && ($2 != "mysql") && ($2 != "test") && ($2 != "information_schema") { print $2 }');
-  for DB in $MYSQL_DBS ; do
-     echo "* Backuping MySQL data from $DB...";
-     mysqldump -h localhost -u root -p$MYSQL_PASS $DB > $DATABASE_EXP/mysql_$DB.sql
-   done
-  echo "-------------------------------------";
-fi
-# Backup and purge old backups
-rdiff-backup --remove-older-than 30B $BACKUP_DIR/$HOST
-rdiff-backup -v 3 --print-statistics --exclude /media --exclude /mnt \
-  --include $OPENKM_HOME --exclude '**' / $BACKUP_DIR/$HOST
-# Start JBoss
-/etc/init.d/jboss start
-echo -e "\n### END: $(date +"%x %X") ###"
-# Status
-echo "=================================";
-rdiff-backup --list-increment-sizes $BACKUP_DIR/$HOST
-echo "*********************************";
-df -h | grep "$BACKUP_DIR"
-echo "=================================";
-# Umount disk
-sync
-umount $BACKUP_DIR
 </source>
-Be sure to have the last JBoss startup script, otherwise use this piece of code to ensure that JBoss is stopped before backuping the files:
+For more information take a look at [http://adminschoice.com/crontab-quick-reference Crontab quick reference]
-<source lang="bash">
+* [[Backup with LFTP]]
-while [ "$(ps -ef | grep java | grep jboss | wc -l)" -gt "0" ]; do
+* [[Backup with rsync]]
-  sleep 5; echo ".";
+* [[Backup with rdiff-backup]]
-done
+* [[Backup with duplicity]]
-</source>
 [[Category: Installation Guide]]
-[[Category: OKM Network]]