TUTORIAL - FULL BACK UP OF ISPCONFIG3 DATA FILES

Discussion in 'Tips/Tricks/Mods' started by ariban99, Jun 23, 2016.

  1. ariban99

    ariban99 Member

    I have gotten so much from the community and ISPCONFIG team, i wanted to share and give back a little.
    I needed a solution for backing up my entire servers data etc... I am running centos 7.
    I paid for a customized backup solution from the amazing guys at schaal.it (Florian Schaal, [email protected]) who build ISPCONFIG, so i would like to share it with everyone else.

    login to your server via ssh or whatever means you want and type:
    cd /root
    nano backup.sh (this will bring a blank screen, copy and and paste the below in it, you can paste in putty by clicking the right mouse key in this blank screen). also note you can change the number of stored backups in line 11 - it`s 3 backups right now (the line that says DOW=`expr $DOW1 % 3` just change teh 3 to any number of copies you want to keep)

    #!/bin/bash
    # Copyright (c) 2016, Florian Schaal, [email protected]
    # All rights reserved.
    #
    # Redistribution and use in source and binary forms, with or without modification,
    # are permitted

    DOW1=`date +%j`
    # no of backups
    DOW=`expr $DOW1 % 3`

    # Path
    BPATH="/var/backup" # like /var/backups no slash at the end!
    WWW="/var/www"
    MAIL="/var/vmail"

    # Databases
    DBISPconfig="dbispconfig"

    # Binaries
    SQLDUMP="/usr/bin/mysqldump"
    SQLBIN="/usr/bin/mysql"
    GZIP="/bin/gzip"
    LEVEL=`date +%u` # day of week (1..7); 1 is Monday
    (( LEVEL-- )) # Mon full-Backup, Sun level 6
    BACKUP_NO=$LEVEL

    # tar-Options
    BACKUP_TAR_OPT_NOZIP="-ch"

    # sonstige
    SYSLOG_TAG=simple-backup.sh
    TMP_LOG=/tmp/backuplog.tmp

    # Funktionen ------------------------------------------------------------------------------------------------
    calculate_time () {
    runtime=`expr $2 - $1`
    HOUR=`date -d @${runtime} "+%H"`
    HOUR=`expr $HOUR - 1`
    MINUTES=`date -d @${runtime} "+%M"`
    SECOND=`date -d @${runtime} "+%S"`
    RUNTIME="0"$HOUR":"$MINUTES":"$SECOND
    SIZE=`ls -lh --si $BPATH/$4|cut -d" " -f5`
    if ([ $4 != NIL ] && [ $SIZE > 0 ])
    then logger -d -t $SYSLOG_TAG $3"-Backup: "$RUNTIME "(size: "$SIZE")"
    else logger -d -t $SYSLOG_TAG $3"-Backup: "$RUNTIME
    fi
    }

    # -----------------------------------------------------------------------------------------------------------
    logger -d -t $SYSLOG_TAG "Backup day $DOW1 - start"
    STARTTIME=`date +%s`

    # save old UMASK
    UMASK=`umask`
    umask 0077

    # /usr-Backup -----------------------------------------------------------------------------------------------
    START=`date +%s`
    logger -d -t $SYSLOG_TAG "/usr backup"
    tar cfz - /usr --exclude=src --exclude=share --exclude=X11R6 --exclude=lost+found --exclude=src --exclude=tmp --exclude=x86_64-suse-linux > $BPATH/$DOW-usr.tar.gz
    STOP=`date +%s`
    calculate_time $START $STOP usr-backup $DOW-usr.tar.gz

    # /var-Backup -----------------------------------------------------------------------------------------------
    START=`date +%s`
    logger -d -t $SYSLOG_TAG "/var backup"
    tar cfz - /var --exclude=adm/autoinstall/cache --exclude=adm/backup --exclude=adm/mount --exclude=adm/YaST/InstSrcManager --exclude=cache --exclude=games --exclude=X11R6 --exclude=lost+found --exclude=lib/clamav --exclude=lib/named/proc --exclude=lib/ntp/drift --exclude=lib/ntp/proc --exclude=lib/ntp/var --exclude=lib/zypp --exclude=lock --exclude=log --exclude=backup --exclude=log --exclude=run --exclude=spool/amavis/tmp --exclude=tmp > $BPATH/$DOW-var.tar.gz
    STOP=`date +%s`
    calculate_time $START $STOP var-backup $DOW-var.tar.gz

    # Complete MySQL-DB-Dump ------------------------------------------------------------------------------------
    START=`date +%s`
    `$SQLDUMP --all-databases --add-drop-table | $GZIP > $BPATH/$DOW-complete_MySQL-DB.sql.gz`
    STOP=`date +%s`
    calculate_time $START $STOP MySQL-DB-Dump $DOW-complete_MySQL-DB.sql.gz

    # ISPConfig-DB ----------------------------------------------------------------------------------------------
    START=`date +%s`
    `$SQLDUMP $DBISPconfig | $GZIP > $BPATH/$DOW-$DBISPconfig.sql.gz`
    STOP=`date +%s`
    calculate_time $START $STOP ISPconfig-DB $DOW-$DBISPconfig.sql.gz

    # mysql-DB --------------------------------------------------------------------------------------------------
    START=`date +%s`
    `$SQLDUMP mysql | $GZIP > $BPATH/$DOW-mysql.sql.gz`
    STOP=`date +%s`
    calculate_time $START $STOP MySQL-DB $DOW-mysql.sql.gz

    # System Backup ---------------------------------------------------------------------------------------------
    START=`date +%s`
    logger -d -t $SYSLOG_TAG "System backup"
    tar cfz - / --exclude=/vz --exclude=/backups --exclude=/dev --exclude=/media --exclude=/data --exclude=/mnt --exclude=/proc --exclude=/sys --exclude=/srv --exclude=/tmp --exclude=/usr --exclude=/var --exclude=/root/.cpan --exclude=/root/backup --exclude=/home > $BPATH/$DOW-system.tar.gz
    STOP=`date +%s`
    calculate_time $START $STOP System $DOW-system.tar.gz
    logger -d -t $SYSLOG_TAG "Backup day $DOW1 - stop"
    exit 0

    SAVE AND EXIT THIS FILE (by pushing control plus X, then it asks if you want to save, click yes) you need root access to save this file.

    next type in ssh:
    chmod 700 backup.sh
    chown root.root backup.sh


    Now we need to edit the my.cnf file, type in ssh:
    nano /root/.my.cnf

    paste the below (CHANGE THE PASSWORD TO BE THE REAL ROOT MYSQL PASSWORD FOR YOUR SERVER)

    [mysql]
    user=root
    password=pass

    [mysqladmin]
    user = root
    password = pass

    [mysqldump]
    user=root
    password=pass

    SAVE AND EXIT THIS FILE (by pushing control plus X, then it asks if you want to save, click yes) you need root access to save this file.

    now type in ssh:
    chown root.root /root/.my.cnf

    add to crontab to automatically run this backup script daily at 4 am (or change the 4 below to any hour you want).
    in ssh type:
    crontab -e

    type i to insert, then paste
    00 4 * * * /root/backup.sh 2>&1 > /dev/null

    Hit esc button to stop insert, then type :x to close and exit

    All the backup files will go into /var/backup folder.

    I have a freenas system, so i also rsync the entire /var/backup folder to my freenas in case my servers ssd drive doesnt work anymore. dont keep your backup on the local server drive only, that is just not smart!!
    hope this helps a lot of people as it did for me!! Enjoy!!

    remember all credit goes to Florian Schaal, [email protected], he did all this for me, i just paid for it!!
     
    nikolaosp, Croydon and till like this.
  2. Croydon

    Croydon ISPConfig Developer ISPConfig Developer

    @ariban99: Thank you for publishing this backup script!

    Small correction: schaal.it makes valuable contributions to ISPConfig, e. g. DKIM and bug fixing, but ISPConfig was initially created and is mainly developed by @till (ISPConfig UG).
     
  3. ariban99

    ariban99 Member

    my bad, sorry!! didnt know that! i am just glad to give something back to this wonderful community of ispconfig.
     
    nikolaosp likes this.
  4. SamTzu

    SamTzu Active Member

    I was just testing this script and it seems to work fine on Perfect Server Debian 8 with Apache.
    There is something on the script that I can't quite figure out though.
    What's with the...

    # sonstige
    SYSLOG_TAG=simple-backup.sh
    It's called here:
    logger -d -t $SYSLOG_TAG "Backup day $DOW1 - start"

    And how would you go about restoring the backups?
     
  5. Fernando Azevedo

    Fernando Azevedo New Member

    I understand this is a very old topic, but did you ever get an answer on how to restore?
     
  6. Jonathon Gilbert

    Jonathon Gilbert Member HowtoForge Supporter

    If there is information on restoring or opening this backup i would be keen to read about it.
     
  7. Jonathon Gilbert

    Jonathon Gilbert Member HowtoForge Supporter

    and has anyone tried this on a ubuntu server installation?
     
  8. Jonathon Gilbert

    Jonathon Gilbert Member HowtoForge Supporter

    How did you get on with this? has it worked ok? and did you manage to figure out how to restore it?
     
  9. Jesse Norell

    Jesse Norell ISPConfig Developer Staff Member ISPConfig Developer

    A quick browse over that script shows it uses tar and mysqldump, with which it creates 3 tar archives per run (one for /usr, one for /var, and one with most of the other system stuff, of which /etc/ and maybe /root are most important) and 3 database dump files (all databases, dbispconfig and mysql). To restore such a backup is a manual process, the general idea is you install a new OS, then extract your backup files somewhere and copy over data (websites and email mostly) to the right location, and restore the sql dumps, all the while you have all your old config (in /etc mostly) to reference as you setup the new server. You do not want to extract the backup files over the top of your new system or you can expect things will likely break. This is relatively straightforward for an experienced administrator; probably a nightmare for beginners. It would help to have installed the original system manually in the first place, to know what goes in to that piece (and I would suggest adding a list of installed packages to the backup for easier reference).

    To find more info, there are other threads about "what to back up" and some (though definitely fewer) about how to restore; it's one of those things you would have to walk through and document as you go to give step-by-step directions, or else you'll surely get some steps out of order, and/or leave something out.

    If you are towards the beginning of your learning curve in system administration, if you have backups similar to this I would recommend just hiring someone to restore the system properly if needed; and if you don't already have backups similar to this, you might look at other backup systems which make the restore piece easier. And just to clarify, I don't mean to imply this is a bad way to perform backups, just that the restore can be more involved (around here we use very similar backups, as well as a backup system with point-and-click restores).
     

Share This Page