Yesterday (it was the first time ever) that the MySQL on one of my ispconfig servers went down. Although, monit has been installed. Monit did not start the MySQL again. Within the log files i could not find something related to the problem. That this could happen was very unplesant. I started mysql again with /etc/init.d/mysql start, but i would like to know how it could happen and why monit did not start mysql (as i expect it to). I have these lines within the file /etc/monit/monitrc for mysql: check process mysql with pidfile /var/run/mysqld/mysqld.pid group database start program = "/etc/init.d/mysql start" stop program = "/etc/init.d/mysql stop" if failed host 127.0.0.1 port 3306 then restart if 5 restarts within 5 cycles then timeout Can anyone advise me how to prevent this problem and tell me what the reason might be.
Did you find any MySQL errors in your logs? You can add this to the MySQL stanza in monit to watch the MySQL socket: if failed unixsocket /var/run/mysqld/mysqld.sock protocol mysql then restart
Thanks Falko, I added that line as well and i hope it will restart mysql if it ever happens again. I try to find the cause within the log files one more time.
MySQL-server went down again This afternoon it happened again that my MySQL-server went down and I don’t know why. This is what I found in the /var/log/syslog: Aug 11 14:32:01 server1 mysqld[30789]: 080811 14:32:01 [Note] /usr/sbin/mysqld: Normal shutdown Aug 11 14:32:01 server1 mysqld[30789]: Aug 11 14:32:03 server1 mysqld[30789]: 080811 14:32:03 InnoDB: Starting shutdown... Aug 11 14:32:06 server1 mysqld[30789]: 080811 14:32:06 InnoDB: Shutdown completed; log sequence number 0 1984718085 Aug 11 14:32:06 server1 mysqld[30789]: 080811 14:32:06 [Note] /usr/sbin/mysqld: Shutdown complete Aug 11 14:32:06 server1 mysqld[30789]: Aug 11 14:32:06 server1 mysqld_safe[25747]: ended Aug 11 14:32:06 server1 mysqld_safe[25818]: started Aug 11 14:32:07 server1 mysqld[25821]: 080811 14:32:07 InnoDB: Started; log sequence number 0 1984718085 Aug 11 14:32:07 server1 mysqld[25821]: 080811 14:32:07 [Note] /usr/sbin/mysqld: ready for connections. Aug 11 14:32:07 server1 mysqld[25821]: Version: '5.0.32-Debian_7etch6-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Debian etch distribution Aug 11 14:32:07 server1 /etc/mysql/debian-start[25859]: Upgrading MySQL tables if necessary. Aug 11 14:32:08 server1 /etc/mysql/debian-start[25871]: Checking for crashed MySQL tables. Aug 11 14:32:53 server1 monit[3598]: 'mysql' process PID changed to 25820 Aug 11 14:32:53 server1 monit[3598]: 'mysql' process PPID changed to 25783 I was able to restart it with /etc/init.d/mysql restart. Monit did not restart MySQL, when MySQL-server went down. How is that possible?
Problem seems to be solved. It seemed that the MySQL threads go up to the maximum value within /etc/my.cnf and then drop. It seems also the reason that Monit could not restart MySQL-server again. By increasing the max_connections setting for MySQL to a higher value, this can be avoided. I hope that this problem is over now. For more monitoring for MySQL, I also changed the MySQL section within /etc/monit/monitrc into: #MySQL check process mysql with pidfile /var/run/mysqld/mysqld.pid group database start program = "/etc/init.d/mysql start" stop program = "/etc/init.d/mysql stop" if failed host 127.0.0.1 port 3306 then restart if failed unix "/var/lib/mysql/mysql.sock" then restart if 5 restarts within 5 cycles then timeout depends mysqld_init depends mysqld_bin depends mysqldsafe_bin check file mysqld_init with path /etc/init.d/mysql group database check file mysqld_bin with path /usr/sbin/mysqld group database check file mysqldsafe_bin with path /usr/bin/mysqld_safe group database I want to thank Till & Falko for their help.
