ISPConfig 3 - back-res a Backup and Restore script

Discussion in 'Plugins/Modules/Addons' started by go0ogl3, Dec 6, 2009.

  1. bseibenick

    bseibenick New Member

    Thank you for a great script!
     
  2. dipeshmehta

    dipeshmehta Member

    Hello all,

    I get
    Code:
    line 238: [: -gt: unary operator expected
    whre the code is
    Code:
    function check_space {
        #pfs=$(($pfs-1)) # for test
        pfs=`df -h $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1`
        #pfs="90"
    
        [B]if [ $pfs -gt $maxp ] ;then[/B]
    	log "There is $pfs% space used on $BACKUPDIR"
    	    if [ $del_en = "yes" ] ; then
    and
    Code:
    line 292: -cjpSPf: command not found
    
    where the code is
    Code:
    if [ -z $fb ] ; then
    	    log "No full backup found for $YX. Full backup now!"
    	    echo > $tmpdir/full-backup$XX.lck
    	    [B]$TAR $ARG $BACKUPDIR/full$XX-$FDATE.tar.bz2 $YX -X $tmpdir/excluded[/B]
    	    log "Backup of $YX done."
    	fi
    I have been using Ubuntu 8.04 Server.

    Please help me to get rid of the situation.

    Thanks in advance.

    Dipesh
     
  3. go0ogl3

    go0ogl3 Member

    Hello, the first error is because the space check fails, one of the $pfs or $maxp parameters is not a number. This is a non fatal error so you can ignore it. Maybe will be corected in a future version of the script, or you can adapt the function to check the free space corectly for your situation... :)

    The second error occurs only if you don't have "tar" installed on your linux, or you don't have "which" command or the "tar" is not in your PATH and is given because the $TAR on the 292 line is NULL. "TAR" is defined in the header of the script by default with TAR=`which tar` on line 111.

    So to get rid of the situation you must check the free space problem if you care about that and most important, you must resolve the "tar" problem. If "tar" isn't in your PATH please define the "TAR=" with the path to the tar binary.

    For more problems, ask here, I'll try to respond ASAP.
     
  4. andypl

    andypl Member

    At the outset thanks for the nice script.
    There is one problem ... me restore everything was full and incremental archive and what if the customer specifically totaled some directories and files? if the script can somehow delete the deleted files or directories?
     
  5. go0ogl3

    go0ogl3 Member

    More info about restore

    The script is unpacking all files to the date requested then is asking you if you want to delete the files created right after the restore date. You see a list of all the files or dir's which will be deleted. If you answer "yes" all files and directories created after the requested restore day are deleted. You can answer no and delete the files or dir's using what pattern you like, after the scripts unpacks all incremental archives.

    You can try the restore part of the script and I recomend everybody to test this part but not directly on /. Use the tmp dir for example. If it's a full linux system you can chroot if you want and test your system before actual restoration.

    Have a nice day!
     
  6. go0ogl3

    go0ogl3 Member

    New version: back-res 0.9.2

    I've managed to improve this script a little, so here is a new version.

    Fixes:
    - First run now does not gives errors
    - Modified the log function to accept first time dir createin
    - Modified the starting sequence to not check the free space if the primary backup directory does not exist
    - If primary backup dir does not exist now it's created at the start
    - Added a line to remove the maildata at the start if the user stops the script before finishing his jobs. This prevents the script to send incorect mails.
    - Added link http://www.howtoforge.com/forums/showthread.php?t=41609
    - Added first TODO

    Thanks goes to nokia80, Snake12, rudolfpietersma, HyperAtom, jmp51483, bseibenick, dipeshmehta, andypl and others

    Allways download the latest version here: http://www.eurosistems.ro/back-res
    Old version: http://www.eurosistems.ro/back-res.0.9.1
     
  7. ikrudolf

    ikrudolf Member

  8. Toucan

    Toucan Member

    I really like the idea of this script and would love to use it. I'm a little confused about the user variables. Is there a sample used by someone else i could see please.

    I'm using debian lenny.

    This is what I tried:

    Code:
    COMPUTER=`cat /etc/badbuntu | awk 'NR==1{print $1}'`    # name of this computer
    DIRECTORIES="/adm /bin /boot /etc /home /lib /root
    /sbin /usr /var"                                        # directories to backup
    EXCLUDED="/bck /tmp /dev /proc /sys /media
    /var/adm /var/cache /var/lib/mysql
    /var/run /var/lock /lib/init/rw /var/tmp
    /var/log/verlihub /var/lib/amavis /var/spool/postfix/p*
    /var/spool/postfix/var *.pid *.lock *.lck"              # exclude those dir's and files
    BACKUPDIR="/mnt/badback/back/$COMPUTER"                   # where to store the backups
    dbuser="root"                                           # database user
    dbpassword="mysqlpass"                                # database password
    email="[email protected]"                             # mail for the responsible person
    TAR=`backuptar`                                         # name and location of tar
    ARG="-cjpSPf"           #sparse                         # tar arguments P = removed /
    EARG="-xjpf"                                            # tar extract arguments P = removed /
    tmpdir="/tmp/tmpbck"                                    # temp dir for database dump and other stuff
    del_en="yes"                                            # Enable delete of files if used space percent > than $maxp (yes or anything else)
    maxp="85"                                               # Max percent of used space before start of delete
    hm="2359"                                               # last minute of the day = last minute of the restored backup of the day restored
    
    
    I tried running it with:
    ./back-res

    and got this result:
    Code:
    badbuntu:/# ./back-res
    cat: /etc/badbuntu: No such file or directory
    ./back-res: line 141: backuptar: command not found
    ./back-res: line 272: [: -gt: unary operator expected
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 298: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    ./back-res: line 332: -cjpSPf: command not found
    badbuntu:/#
    
    Any help much appreciated
     
    Last edited: Apr 21, 2010
  9. SimonH

    SimonH New Member

    that line in your script should probably just be

    Code:
    TAR=`which tar`
    Great script by the way go0ogl3, works perfectly for me, I just need to tag an scp command on the end now to send the backup to a remote box.

    Nice job ;)
     
  10. Toucan

    Toucan Member

    Thanks Simon. Works great now thanks.

    For the benefit of others who are at my level of learning, using this line:

    Code:
    40 3 * * * /etc/back-res 1>/dev/null 2>/dev/null
    in the crontab will send all output into the 'bit bin'. That means where there were problems such as trying to backup a non existent directory it'll carry on with no offputting error messages to the user. It'll just run during the night with a nice surprise for the morning.

    In debian lenny you can find the crontab: /etc/crontab then use your favourite editor to add that line.
     
  11. nokia80

    nokia80 Member

    hello all

    I suddenly get a strange error what can I do about it?

    / Bin / tar: / adm: Can not stat: No such file or directory
    / Bin / tar: Error exit delayed from previous errors


    best regards michael nokia80
     
  12. Toucan

    Toucan Member

    You've not mentioned what distro you are using. If you use debian, like me, it does not have the directory /adm. Remove that from back-res user variables.
     
  13. nokia80

    nokia80 Member

    hello

    thanks for the reply yes it's debian that error is gone now but now have this error
    cat: / Is a directory etc/ns1 .****:**


    best regards michael
     
  14. Toucan

    Toucan Member

    how are you calling the script?

    edit the /etc/crontab

    and add the line:
    Code:
    40 3 * * * /etc/back-res 1>/dev/null 2>/dev/null
    This will call the script at 40 minutes past three each day and will send any error messages to the bit bin.

    Check to see if your script is in fact making the back up tars in your designated directory regardless of the errors it's giving. I understand some errors aren't critical.
     
  15. nokia80

    nokia80 Member

    hi toucan

    i cal the script like this ./back-res




    michael
     
  16. Toucan

    Toucan Member

    Check to see if the script is in fact making the tars.

    If it is call it from you crontab using a line like this:

    Code:
    40 3 * * * /etc/back-res 1>/dev/null 2>/dev/null
    This part of the line sends any output (such as error messages) to the 'bit bin'. That way you won't see them. As long as it does it's job then all is good.

    I'm no expert on this, I can only quote from what works for me from advice from others so feel free to step in and comment anyone who knows better.
     
  17. nokia80

    nokia80 Member

    hi all

    I have this code in 40 3 * * * / etc / backup-res 1> / dev / null 2> / dev / null cronetab done.

    but nothing happens am I doing something wrong or he must be a single line

    he sits or in the proper file my server is running debian.


    best regards michael
     
  18. Toucan

    Toucan Member

    if you have vim as an editor:
    Code:
    vi /etc/crontab
    add the line:
    Code:
    40 3 * * * /etc/back-res 1>/dev/null 2>/dev/null
    You had spaces in yours and i don't know if that was intentional. Copy and paste the above line.

    Make sure that script 'back-res' is in the file :

    Code:
    /etc
    and is called 'back-res'
     
  19. dipeshmehta

    dipeshmehta Member

    Sorry go0ogl3, after your making available the improved script, I couldn't get enough time to check it. Today, I checked and as expected it works like a charm. Thanks for the great work done.

    Moreover, I came to know one point that the error
    Code:
    line 238(or 273): [: -gt: unary operator expected
    might be due to wrong information store in
    Code:
    COMPUTER=`cat /etc/HOSTNAME | awk 'NR==1{print $1}'`
    variable. As I have been using ubuntu, my hostname store in /etc/hostname (not /etc/HOSTNAME). If $COMPUTER variable has wrong data or empty, then $BACKUPDIR will point wrong, recursively pfs=`df -h $BACKUPDIR | awk 'NR==2{print $5}' | cut -d% -f 1` shall also have wrong data, giving error on line 238 (or 273).

    Those who get such error, check COMPUTER=`cat /etc/HOSTNAME | awk 'NR==1{print $1}'` line and alter according to your distro.

    Thanks once again go0ogl3, this is exactly what I wanted for backup solution.

    Dipesh
     
  20. go0ogl3

    go0ogl3 Member

    This error may be because you don't have the /adm directory. Please check the "DIRECTORIES" variable (directories to backup) in the begining of the script.

    Thank you all!
     

Share This Page