This evening one of my servers, which happens to be a slave filled it's hard disk up by mistake causing the server to grind to halt. In an attempt to free up the disk space I set the master to delete a number of test sites. However, because the slave was in such a bad way it didn't respond and now a number jobs are sat in the job queue waiting to process. After freeing up disk space the slave has now returned to normal operation, however there are still the same jobs stuck in the queue. I can now SSH into the slave without problem. Is there a way I can prompt the pair to start the jobs again?
Sorry, yes should have given you a bit more information. Debian Lenny ISP Config3.0.4.6 ok, set the master to debug, commented out the line and ran /usr/local/ispconfig/server/server.sh and got this back at the command line: Code: badbuntu:~# /usr/local/ispconfig/server/server.sh 13.10.2012-22:05 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 13.10.2012-22:05 - DEBUG - No Updated records found, starting only the core. /usr/bin/fail2ban-client /sbin/iptables /sbin/ip6tables 13.10.2012-22:05 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock finished. I did the same on the slave and this time now it updated all the records and cleared the jobs. So that's great! The only problem I have now is the monitor is still not updating. The slaver server is still shown as critical and in red showing mysql offline, although it mysql is in fact online. The most recent entries in the log is looking like this: Code: 2012-10-13 22:19 badbison.co.uk Debug Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:19 badbison.co.uk Debug No Updated records found, starting only the core. 2012-10-13 22:19 badbison.co.uk Debug Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:19 badbison.badbison.com Debug Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:19 badbison.badbison.com Debug No Updated records found, starting only the core. 2012-10-13 22:19 badbison.badbison.com Debug Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:19 badbison.badbison.com Debug Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:19 badbison.badbison.com Debug No Updated records found, starting only the core. 2012-10-13 22:19 badbison.badbison.com Debug Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:18 badbison.co.uk Debug Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:18 badbison.co.uk Debug No Updated records found, starting only the core. 2012-10-13 22:18 badbison.co.uk Debug Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:16 badbison.co.uk Debug Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 2012-10-13 22:16 badbison.co.uk Debug No Updated records found, starting only the core. 2012-10-13 22:16 badbison.co.uk Debug Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock So the only real problem left is to get the monitor to update itself? Thanks
Then the cron daemon was not running on the slave. The monitor updating its records at specific times, so you can not update it by running the script manually if you dont do this at the exact minute were its processing a monitor job. When you start your cron daemon again, then it will update the monitor automatically after some time.
A similar problem seems to have come back where jobs are not processing on the slave... running the script manually, the slave complains it cannot connect to the master: Code: 25.10.2012-00:45 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock 25.10.2012-00:45 - WARNING - Unable to connect to master server. 25.10.2012-00:45 - DEBUG - No Updated records found, starting only the core. /usr/bin/fail2ban-client /sbin/iptables /sbin/ip6tables 25.10.2012-00:45 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock finished.
This means that it is not possible to connect frm slave to master mysql database with the login details for the master database from config.inc.php. Reasons can be: Firewall Changed hostname Changed mysql login details Mysql not listening on the external network interface on the master Changed ip address
Thanks Till I restarted the slave and then ran the script again but it didn't seem to make any difference. I restarted the master and then suddenly the jobs seem to start processing. I'll keep an eye on this.
