SVN: web backups

Discussion in 'Developers' Forum' started by tom123, Jul 30, 2012.

  1. tom123

    tom123 Member

    I'm looking at the latest SVN trunk.
    How can web backup works in a multi server setups?
    The client doesn't have write access to sys_datalog and web_backup tables so the following query will always fail:

    Code:
    //* Insert web backup record in database
    $insert_data = "(server_id,parent_domain_id,backup_type,backup_mode,tstamp,filename) VALUES (".$conf['server_id'].",".$web_id.",'web','".$backup_mode."',".time().",'".$app->db->quote($web_backup_file)."')";
    $app->dbmaster->datalogInsert('web_backup', $insert_data, 'backup_id');
    
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    About which file / line are you talking?
     
  3. tom123

    tom123 Member

    cron_daily.php
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Ok, I will check that. The new permission scheme is not added yet to the installer, so it might be that some tables are not accessible yet.
     
  5. tom123

    tom123 Member

    I've added SELECT,INSERT,DELETE grants to web_backup table, but I don't see any grant for sys_datalog.

    Is sys_datalog supposed to be different between master and client ?
    Actually, backup entries are stored on the client only.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    I've added a report for the bugtracker for this and we will review it until 3.0.5 gets released. The sys_datalog is used to sync information between master and client and the information about the backups are needed o master and client.
     
  7. tom123

    tom123 Member

    Ok, but sys_datalog can be different between master and client ?
    The actual cron will write only to the client, seems to work because i'm able to see the backup list from the interface, but on the master there is no reference to these backups.

    I don't know why master should have the client backup list, maybe that the actual script is good as it is.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes. The sys_datalog of the master contains all records for all clients while the sys_datalog of a client contains only record that are required by this specific client.

    And thats the reason why the code writes to the dbmaster and not the local db api, if you change it to the local db, you cant see any reference on the master.
     
  9. tom123

    tom123 Member

    Ok, in this case, the backup script is working properly because the client calls
    Code:
    $app->dbmaster->datalogInsert('web_backup', $insert_data, 'backup_id');
    
    datalogInsert will do the insert and then calls datalogSave that will write on "db" and not on "dbmaster".

    In this way, the backup item is wrote properly on sys_datalog on the client and never on the master.

    If this is correct, cron script is working properly.
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    There is a error in the mysql library indeed, thansk for pointing that out. Ine the datalogSave function, it has to be:

    $this->query($sql);

    instead of:

    $app->db->query($sql);

    and the two quote calls should be also to $this. I fixed that in svn. It currently works but it was not intended this way and I had not noticed it cause it worked. I will check the issue regarding permissions.
     
  11. tom123

    tom123 Member

    did you changed only the datalogSave function?
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, the other functions used already $this-> as far as I've seen.
     
  13. tom123

    tom123 Member

    sys_datalog on master is still not updated by client's cron.
     
  14. tom123

    tom123 Member

    With SVN changes, even the insert on web_backup on master doesn't work anymore. Our permissions are correct.
     
  15. tom123

    tom123 Member

    The same broken query apply on mail stats calculation.
    From your daily_cron.php, you are calculating mail traffic saving data to the master db:

    ($size is one on mine customization, there is no $size in SVN code)


    Now, from the interface code (mail_user_stats.php), you are quering the local server and not the master:

    Code:
    	function prepareDataRow($rec)
        {
    		global $app;
    		
    		$rec = $app->listform->decode($rec);
    
    		//* Alternating datarow colors
    		$this->DataRowColor = ($this->DataRowColor == '#FFFFFF') ? '#EEEEEE' : '#FFFFFF';
    		$rec['bgcolor'] = $this->DataRowColor;
    		
    		//* Set the statistics colums
    		//** Traffic of the current month
    		$tmp_date = date('Y-m');
    		$tmp_rec = $app->db->queryOneRecord("SELECT traffic as t FROM mail_traffic WHERE mailuser_id = ".$rec['mailuser_id']." AND month = '$tmp_date'");
    		$rec['this_month'] = number_format(intval($tmp_rec['t'])/1024/1024, 0, '.', ' ');
    		
    		//** Traffic of the current year
    		$tmp_date = date('Y');
    		$tmp_rec = [B]$app->db->queryOneRecord[/B]("SELECT sum(traffic) as t FROM mail_traffic WHERE mailuser_id = ".$rec['mailuser_id']." AND month like '$tmp_date%'");
    		$rec['this_year'] = number_format(intval($tmp_rec['t'])/1024/1024, 0, '.', ' ');
    		
    
     
  16. tom123

    tom123 Member

    News about this?
     
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    Thats correct and intended. The stats are only shown on the master so they are saved on the master. If you would save them on the slave, nobody would be able to see them.

    Thats correct as well. The only server that runs a interface in a ISPConfig cluster is the master servers, so when you query the local database in the interface, the amster is queried.
     
    Last edited: Aug 6, 2012
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    See #6 in this thread.
     
  19. tom123

    tom123 Member

    What I still don't understand is:

    - is ISPConfiged engineered to have different datas between master and client databases? Actually, some data (mostly coming from crontab) are saved only on master, this will make an inconsistency on the slave itself.

    Is this ok?

    For example, mail usage statistics are generated by slave crontab and saved on master DB. This will make a different mail_traffic table between master and slave. The slave will be always empty. Is this ok?
     
  20. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, ISPConfig is enginered to store only the nescessary data on a node and not all. The database of a slave contains only the data that is required on that slave locally.

    Thats ok. Saving data on a server were this data is not needed for anything makes just no sense. So the statistics data which is only needed on the master is only saved on the master.
     

Share This Page