install monit

I have succesfully installed munin and I am now procceeding to install monit. For monit, I am following this guide: http://www.howtoforge.com/server_monitoring_monit_munin_p2

1) If monit will monitor the services, restart them if needed and message the admin if something is wrong, right? But I thought that ispconfig also does this:

Maybe I have to dissable this function in ispconfig?

2) These messages are sent to some email? Via postfix? Because I am trying to strip the server down. I do not use any email functionality, so I have removed amavis, clamav (spam, antivirus etc).

3) I want to forward all email warning to my personal email account, eg [email protected]. How do I do this?

 

So I UNCHECK "Enable service monitoring and restart on failure" and I CHECK the other 3 options, right?

 

When I start MONIT, I am getting the error:

Code:

/etc/monit/monitrc:59: Error: syntax error 'localhost'

Here are lines 58 and 59 from /etc/monit/monitrc:

Code:

set mailserver smtp.gmail.com port 587 username "[email protected]" password "xxxxxx" using tlsv1 with timeout 30 seconds,   # primary mailserver
               localhost                    # fallback relay

Here I try to first use GMAIL to send the mail. If this falis, then it will use postfix from localhost, correct? What is wrong with the syntax (I found this in a guide).

And later on I have this code to send messages to both these addresses:

Code:

set alert root@localhost
set alert [email protected]

 

I checked the official syntax and then tried to remember from the University days: http://mmonit.com/monit/documentation/monit.html

Code:

 SET MAILSERVER {hostname|ip-address [PORT port]
                [USERNAME username] [PASSWORD password]
                [using SSLV2|SSLV3|TLSV1] [CERTMD5 checksum]}+ 
                [with TIMEOUT X SECONDS]
                [using HOSTNAME hostname]

The following did not give me any error messages:

Code:

set mailserver smtp.gmail.com port 587 username "[email protected]" password "xxxxxx" using tlsv1,    # primary mailserver
               localhost                    # fallback relay
               with timeout 30 seconds

Obviously, the "with timeout 30 seconds" had to go at the end, after all mail servers.

Thank you for the brainstorming. It was good that you reasured me I was on the right track and not totally wrong.

 

Till

Since I use nginx 1.4.4 instead of Apache, can you help me rewrite the nginx part of /etc/monit/monitrc ?

The tutorial (http://www.howtoforge.com/server_monitoring_monit_munin_p2) says:

Code:

check process apache with pidfile /var/run/apache2.pid
   group www
   start program = "/etc/init.d/apache2 start"
   stop program  = "/etc/init.d/apache2 stop"
   if failed host www.example.com port 80 protocol http
      and request "/monit/token" then restart
   if cpu is greater than 60% for 2 cycles then alert
   if cpu > 80% for 5 cycles then restart
   if totalmem > 500 MB for 5 cycles then restart
   if children > 250 then restart
   if loadavg(5min) greater than 10 for 8 cycles then stop
   if 3 restarts within 5 cycles then timeout

I managed to write this:

Code:

check process nginx with pidfile /var/run/nginx.pid
start program = "/etc/init.d/nginx start"
stop program = "/etc/init.d/nginx stop"
if failed host 127.0.0.1 port 80 then restart
if cpu is greater than 40% for 2 cycles then alert

Fot a test file, I have made https://surf-anonymous.info/test/index.html

Also, what am I supposed to enter for email address here (/var/certs/monit.cnf):

Code:

emailAddress_default            = [email protected]

After starting MONIT, I started getting to my gmail aggress these emails:

Code:

Delivery to the following recipient failed permanently:

     root@localhost

Technical details of permanent failure:
DNS Error: Domain name not found

Most probably, at some configuration file I entered root@localhost, while it should be a more meaningfull [email protected]?

I am also getting these emails:

Code:

Monit instance changed Service myhost.mydomain.tld

        Date:        Wed, 22 Jan 2014 23:56:13
        Action:      start
        Host:        server1.surf-anonymous.info
        Description: Monit started

Your faithful employee,
Monit

Do I have to replace "myhost.mydomain.tld" with something else at some configuration file too?

 

ok, I inserted the nginx section. monit is working and it does monitor nginx.

issue 1: When I reset the VPS, I got the following undelivered emails (along with the correct ones delivered at [email protected]):

Code:

Delivery to the following recipient failed permanently:

     root@localhost

Technical details of permanent failure:
DNS Error: Domain name not found

This is the code I use in monitrc:

Code:

# send alert emails to both addresses 
set alert root@localhost
set alert [email protected]

I copied "set alert root@localhost" from the guide. What am I doing wrong here?


issue 2:
How do I monitor php5-fpm too?

I copied the following code but it does not work because this is invalid "/tmp/php-fpm.sock":

Code:

check process php5-fpm with pidfile /var/run/php5-fpm.pid
   group php 
   start program = "/etc/init.d/php5-fpm start"
   stop program  = "/etc/init.d/php5-fpm stop"
   if failed unixsocket /tmp/php-fpm.sock then restart
   if 4 restarts within 5 cycles then timeout
   depends on nginx

After searching the configuration files, I found the following files in this directory "/var/lib/php5-fpm/":
web2.sock
ispconfig.sock
apps.sock

issue 3:
When things start to fail, I'd like to REBOOT the server. How do I do this?

I did copy the commands like "if 4 restarts within 5 cycles then timeout", but the manual would not tell me what this "timeout" action does really. What I really want is :
if 4 restarts within 5 cycles then REBOOT THE SERVER

 

issue 1:
I made several changes, I still have the same problem. How can I see if root@localhost is setup correctly to receive emails?

contents of my HOSTS file:

Code:

127.0.0.1 localhost.localdomain localhost
# Auto-generated hostname. Please do not remove this comment.
192.210.214.129 server1.surf-anonymous.info  server1
::1 localhost.localdomain localhost

Code:

root@server1:~# sudo postfix status
postfix/postfix-script: the Postfix mail system is running: PID: 706

Code:

root@server1:~# sudo postconf mydestination
mydestination = server1.surf-anonymous.info, localhost, localhost.localdomain

So I decided to reinstall postfix, by following this guide: https://www.digitalocean.com/community/articles/how-to-install-and-setup-postfix-on-ubuntu-12-04

I created this file: /etc/postfix/virtual
tha contains the following line:
[email protected] root

I tried it and again I have the following message from monit:

Code:

Delivery to the following recipient failed permanently:

     root@localhost

Technical details of permanent failure:
DNS Error: Domain name not found

does this help?

Code:

root@server1:~# grep -r -l "root@localhost" /etc/*
grep: /etc/blkid.tab: No such file or directory
/etc/fail2ban/jail.conf
grep: /etc/mail/smrsh/mail.local: No such file or directory
/etc/mail/submit.cf
/etc/monit/monitrc
/etc/warnquota.conf

issue 2:
I have ubuntu 12.04. This file does not exist: /var/run/php5-fpm.sock

issue 3:

You mean to replace this

Code:

if 3 restarts within 5 cycles then [B]timeout[/B]

with this?

Code:

if 3 restarts within 5 cycles then [B]reboot[/B]

 

issue 3:

How come the "reboot' command is not mentioned in the monit manual? http://mmonit.com/monit/documentation/monit.html

What I have is this:

Code:

check process nginx with pidfile /var/run/nginx.pid
start program = "/etc/init.d/nginx [B]start[/B]"
stop program = "/etc/init.d/nginx stop"
if failed host 127.0.0.1 port 80 then restart
if cpu is greater than 40% for 2 cycles then alert

You suggest that I change it to this:

Code:

check process nginx with pidfile /var/run/nginx.pid
start program = "/etc/init.d/nginx [B]reboot[/B]"
stop program = "/etc/init.d/nginx stop"
if failed host 127.0.0.1 port 80 then restart
if cpu is greater than 40% for 2 cycles then alert

From what I understand, the effect of this is:
1) if nginx is not working it will reboot the entire server
2) if nginx is loaded but failing to respond properly (if failed host 127.0.0.1 port 80 then restart) then reboot the server.

Is this what you mean?

Well, I do not want #1 to happen. If a process is not loaded, I want it to load (START) without rebooting. If several restarts fail, then something is wrong with the server so I want the server rebooted. This is why I was thinking of something allong the lines of

Code:

if 3 restarts within 5 cycles then reboot

Obviously this code does not work. But if you understand what I am trying to do, what would be the command for this?

 

So you are suggesting this:

Code:

check process nginx with pidfile /var/run/nginx.pid
   start program = "[B]reboot[/B]" 
   stop  program = "/etc/init.d/nginx stop"
   if failed host 127.0.0.1 port 80 then restart

Obviously I do not want this.

I will first try to start the service. So the following code is good, for what it does:

Code:

check process nginx with pidfile /var/run/nginx.pid
   start program = "/etc/init.d/nginx start"
   stop  program = "/etc/init.d/nginx stop"
   if failed host 127.0.0.1 port 80 then restart

What I originally asked you is how can I reboot the server if nginx or any other process repeatedly fails after many restarts. Something like this:

Code:

check process nginx with pidfile /var/run/nginx.pid
   start program = "/etc/init.d/nginx start"
   stop  program = "/etc/init.d/nginx stop"
   if failed host 127.0.0.1 port 80 then restart
   if 4 restarts within 5 cycles then [B]reboot[/B]

Well, I just tried it, it does not work:

Code:

root@server1:~# monit -t
/etc/monit/monitrc:157: Error: [B]syntax error 'reboot'[/B]

So how do I do this?

 

issue 3:

ok, "monit -t" did not complain about it. Now I will have to test it. Thank.

issue 2:
I am still stuck with monitoring php5-fpm. I do not know how to test if it is working. It might work if I could find the file php-fpm.sock.

 

issue 3:

I went to file /etc/php5/fpm/pool.d/www.conf and I replaced this line

Code:

listen = 127.0.0.1:9000

with this line:

Code:

listen = /var/run/php5-fpm.sock

After restarting php5-fpm the unix socket file was created for php5-fpm here:

Code:

/var/run/php5-fpm.sock

More details here: bbb-solutions.blogspot.com/2014/01/where-is-ubuntu-php5-fpm-unix-socket.html

After that it was trivial to configure monit:

Code:

 check process php5-fpm with pidfile /var/run/php5-fpm.pid
   group php 
   start program = "/etc/init.d/php5-fpm start"
   stop program  = "/etc/init.d/php5-fpm stop"
   if failed unixsocket /var/run/php5-fpm.sock then restart
   if 4 restarts within 5 cycles then exec /sbin/reboot
   depends on nginx

It looks easy after you know what you are doing.

Thanks Till