Extending Perfect Server - Debian Squeeze [ISPConfig 3]

As I see my tutorial is posted. Please, whoever follow it, MUST be very careful. Especially the page 5 with the firewall.

The page 6 is just a suggestion for the ISPConfig developers, and no a final solution. The logic is to give the client a full back of his data IN his directory. I know that this means that his space must be bigger, but I think that this is worth it.

 

I understand. That was my first thought, but I saw duplicate backups in the web folders (in cases of alias etc). That time I didn't check in the sql query if the domain was an alias, or a 'real' domain. Now I think I can prevent that.

I will try to follow your advice and change the storing folder. I will post here the results.

As of webmin, I totally agree with you. I only install it for web based access in case of problems (e.g. I once managed to lock myself out of ssh. Thank God, I had installed webmin, and I unlocked myself.) Probably I must mention it in the tutorial.

P.S.
Could you move this thread to the right section of forum? I accidenty put it here, when it should be placed to 'How To related"

 

Why do you say that the 'client' directory is not accessible?

I have a client webxy that can access '/var/www/clients/clientz/' directory (through sftp).
I followed the instructions in the manual and Perfect Server and the client can download his backup as long as the permissions on the files are ok for him (660 for webxy:clientz).

Am I missing sth here?

I do not have jail enabled as I haven't solved the problem with the greek characters yet ( http://www.howtoforge.com/forums/showthread.php?t=51047 )

 

I now know what you mean
I always use ssh (sftp) to access my files, and of course (assumed that jail isn't enabled) I can access the whole filesystem
With ftp, my root dir is the clientz/webxy folder.
I will change the script accordingly.
Thanks.

 

Ok. I did it.
Now the backups (websites and dbs) are stored in the webxy dir.
After a very few tests, I see that there are no dublicates (concerning clients with alias domains etc).
I can also access them through ftp, as you mentioned.

I updated the tutorial, too.

Thanks again Till.

 

8omas,

Thanks for this How-To. A lot of great things in it. I've been able to get most of it working but there are a couple parts where I'm getting error messages I haven't been able to trouble shoot.

When running /root/scripts/mytail I get:

and when I try /root/scripts/mybackup.sh I get:

 

As for the mytail, you probably forget the \ (backslash) at the end of the line:
I updated the tutorial, so it is more readable:

Code:

#!/bin/bash
multitail -ci yellow -e "ailed" -n 1000 /var/log/auth.log  \
-ci red -e "Ban" -n 1000 -I /var/log/fail2ban.log \
-ci red -e "[COLOR="Red"]fw[/COLOR]" -n 1000 -I /var/log/messages \
-ci green -e "Unban" -n 1000 -I /var/log/messages \
-ci blue -e "fail" -n 1000 -I /var/log/syslog

please change the word iopen with fw.

In my backup, there is a mistake:

Code:

QRY="use dbispconfig; SELECT web_domain.system_user, web_domain.system_group, \
web_domain.document_root, web_domain.domain FROM web_domain WHERE  \
web_domain.type!='alias' AND web_domain.system_user IS NOT NULL AND LENGTH(web_domain.redirect_path)<5 OR web_domain.redirect_path IS NULL) ;"

must be written as:

Code:

QRY="use dbispconfig; SELECT web_domain.system_user, web_domain.system_group, \
web_domain.document_root, web_domain.domain FROM web_domain WHERE  \
web_domain.type!='alias' AND web_domain.system_user IS NOT NULL AND (LENGTH(web_domain.redirect_path)<5 OR web_domain.redirect_path IS NULL) ;"

I corrected this.
Thanks

 

Sorry, I just saw that is a copy-paste mistake.
The same mistake appears in all queries.
Just change the NULL) to NULL and you 'll be ok

Not correct. See the next post

 

The mistake is the missing ( before LENGTH -- NOT what I mention above. So you must have this:

Code:

(LENGTH(web_domain.redirect_path)<5 OR web_domain.redirect_path IS NULL)

I reupdated the tutorial

sorry for the inconvenience

 

8omas,

Thanks for the quick updates, they work great now.

There is one behavior of the backup script that I'm not 100% happy with though. Say I have 1 client with 4 databases and 10 sites. Then when the backup script runs, I get 4 backed up database files in each of the 10 site folders along with the backup files for that site. So instead of 4 backup files for the 4 databases, there are 40.

I think it would work better if the files were stored at the client level in a backup folder and not at the site level. I just don't know what to think about the permissions and access to them via ftp...

~Taz

 

As I see, there are still duplicates. Unfortunately, I cannot find any solution for this for now. This is because, there are no data in the database of ISPConfig that associate databases with sites.

Till, Falko (or any other developer), you made a schema that is website-centric but the databases are owned by clients and (databases) are not associated with websites. This means that the 'system' has no knowledge of which database is used by which website. I know that this gives access to client to all his databases, but shouldn't it be the same for his websites? Isn't it meaning that the db logic is client-centric and not website-centric?
Am I right?

If I am, would it be possible to have a field (for every database) to optionally associate it (or even mandatory) with the web site?

If the previous suggestion is not possible, would it be logical to create an ftp user for the client to have access to /var/www/clients/clientxy through ISPConfig interface? (From the advanced settings in ftp user I see that we can change his path to access all his websites)

 

I am not happy too. As you see I tried to change this, but this is not possible since there are no data that associates databases with websites (I hope I am mistaken but as much as I searched the ISPConfig db I couldn't find anything)

My first approach was to store files in client directory, but Till suggested not to do this.

My suggestion for now is to improve the script so as to backup websites to web folders and databases to client folders.

This means two alternative things
1) that you have to login 2 times, with 2 different ftp accounts (one for the client and one for the website owner - ftpuser -- see my previous post )
2) or that you have to login through sftp (ssh) and download the files.

In each case I must change the script and in each case someone has to have access to /var/www/clients/clientXY

I will try with a new script in the afternoon.

Thank you very much for your comments

 

Because the backups are not accessible through ftp.

How are they accessible if they login? Through the panel?

We have already talked about this in those threads:
http://www.howtoforge.com/forums/showthread.php?t=51723
http://www.howtoforge.com/forums/showthread.php?t=51586

I think that the only viable solution is to create an ftp user with access to all his websites (changing his path). Why is this bad? The sites and the db belong to him, and he is not a reseller. I did a small test and I thinks it works.

 

If an ftp user has access to /var/www/clients/clientXY, he cannot store (upload) files. But as I tested he can download them. So...

I could do 2 things, until the new release:
1) revert the script to previous version, and give one or all ftp accounts of the client, access to client folder (having in mind that he cannot upload -- I think that it has to do with the ftp server). With this he could do whatever he would do and have no problem (I think)
2) Keep the script almost as is. I will remove the db part and maybe just store all db's in one site

Any suggestion?
I tend for the second one.

One more thing. A lot of clients tend to use the same db for a lot of sites. The association isn't the best solution even though a lot of panels use this approach. This looks like the second of the above with the only difference that all the dbs are in one site.

Last. Is it worth trying to solve this one, or the new version will be released soon?

 

Thank you all for your comments.

The solution I followed is the one that all the databases are stored in one site. In case, a client has more than one database, then all the databases will be backed up in his first (based on webID) site.

I updated the tutorial, so you can empty the local script (mybackup.sh) file and paste all the lines from the manual.

If you want you can just replace the second query with the following (you just loose some comments):

Code:

QRY="use dbispconfig; SELECT web_database.database_name , web_database.database_user ,\
 min(web_domain.system_user) as muser, web_domain.system_group, min(web_domain.document_root) as mpath, \
web_domain.domain FROM web_database, web_domain WHERE web_database.sys_userid=web_domain.sys_userid \
AND web_database.sys_groupid=web_domain.sys_groupid AND web_domain.type='vhost' \
AND web_domain.system_user IS NOT NULL AND (LENGTH(web_domain.redirect_path)<5 OR web_domain.redirect_path IS NULL) \
 GROUP BY web_database.database_name , web_database.database_user,  web_domain.system_group;"

Don't worry about the duplicated dbs in websites' folders. They will be removed at the next run. In you main folder you don't have duplicated dbs.

Please let me know, if everything is ok now.

 

Hmmmm ... finally you had to worry about duplicate files .
I had some mistakes with the rm and the wildcards.

instead of

Code:

                [ -f ${col[2]}/*BU.tar.gz ] && rm ${col[2]}/*BU.tar.gz 
                [ -f ${col[2]}/*BU.gz ] && rm ${col[2]}/*BU.gz 

you must have

Code:

                for delfile in ${col[2]}/*BU*gz ; 
                do [ -f $delfile ] && rm $delfile; 
                done

The same happens in one more place :

Code:

                [ -f $SITES/*.$BACK22.gz ] && rm $SITES/*.$BACK22.gz
		[ -f $SITES/*.$BACK22.tar.gz ] && rm $SITES/*.$BACK22.tar.gz

where instead should be:

Code:

                for delfile in $SITES/*$BACK22*gz ; 
                do [ -f $delfile ] && rm $delfile;  
                done

Please copy, all the new code from the 6th page of the manual.
I am talking about mybackup.sh