Monit window blank in ISPConfig web GUI

I have a nice solution for apache that I should dig up and post. I started writing a tutorial for "everything" to setup beyond the debian 9 perfect server guide and can see that was a mistake, as the scope is way to big to complete, and in practice is never complete; some solutions and code exists nowhere else but that unfinished tutorial, and now I have almost no debian 9 servers left to reference in order to do so. The apache config and the monit config is pretty straightforward, though i think it relied on a lets encrypt certificate for the server which was setup and managed in separate places, so can't just cut&paste a solution as quickly. I'll try to get it put together sometime and post it.

 

@Jesse Norell: Maybe split it into several small tutorials / topics?

 

I was going to try a more complete guide for just the monit/apache setup, but it's not quick, so here are some pieces you can work with. This should configure apache to proxy /monit/ to the local monit server on port 2812. Monit authenticates the apache requests by means of a client certificate, which has to be created and maintained. Apache authenticates the user by checking for a valid ispconfig login session, so you can only view /monit/ when you are logged in to ispconfig.

In monit config, you'll need something like:

Code:

set httpd port 2812 and
    use address localhost
    ssl enable
    pemfile /etc/ssl/private/monit.pem
    clientpemfile /etc/ssl/private/monit-clientpemfile.pem
    allow localhost

set ssl {
    verify     : enable,
    selfsigned : allow    # slave servers are using self-signed certs for monit
}

The monit related certificate files are maintained in some other scripts, but the relevant pieces are:

Code:

MONIT_CERT=/etc/ssl/private/monit.pem
MONIT_CLIENTPEMFILE=/etc/ssl/private/monit-clientpemfile.pem
MONIT_APACHE_CERT=/etc/ssl/private/apache-proxymachine.pem

LE_DIR=/etc/letsencrypt/live/$(hostname -f)
LE_CHAIN=${LE_DIR}/chain.pem
LE_CERT=${LE_DIR}/cert.pem
LE_FULLCHAIN=${LE_DIR}/fullchain.pem
LE_KEY=${LE_DIR}/privkey.pem

function get_issuer_certs() {
  for F in ${@}
  do
    if [ -f ${F} ]; then
      echo -n "${F} "
      F_HASH=$(${OPENSSL} x509 -hash -noout -in ${F})
      F_ISSUER_HASH=$(${OPENSSL} x509 -issuer_hash -noout -in ${F})
      if [ "${F_HASH}" != "${F_ISSUER_HASH}" ]; then
        get_issuer_certs ${F_ISSUER_HASH}
      fi
    elif [ -f /etc/ssl/certs/${F}.0 ]; then
      get_issuer_certs /etc/ssl/certs/${F}.*
    else
      echo "get_issuer_certs: unknown input: ${F}" 1>&2
    fi
  done
}

LE_ISSUER_HASH=$(${OPENSSL} x509 -issuer_hash -noout -in ${LE_CHAIN})
LE_ISSUER_CERTS=$(get_issuer_certs ${LE_ISSUER_HASH})

# copies/formats the letsencrypt files for monit
function setup_cert_monit() {
  # this is the certificate used by monit httpd
  cat ${LE_KEY} ${LE_FULLCHAIN} > ${MONIT_CERT}
  ${OPENSSL} dhparam 1024 >> ${MONIT_CERT} 2>/dev/null
  chown root:root ${MONIT_CERT}
  chmod 600 ${MONIT_CERT}

  # this is the pemfile to verify clients, both apache and monit cli
  ${OPENSSL} rsa -in ${LE_KEY} -out ${MONIT_CLIENTPEMFILE}.key 2>&1 | grep -v 'writing RSA key' 1>&2
  cat ${MONIT_CERT} ${LE_ISSUER_CERTS} ${MONIT_APACHE_CERT} > ${MONIT_CLIENTPEMFILE}
  rm -f ${MONIT_CLIENTPEMFILE}.key
  chown root:root ${MONIT_CLIENTPEMFILE}
  chmod 600 ${MONIT_CLIENTPEMFILE}
}

Something then should run whenever the letsencrypt certificate for the server is updated to rebuild this and restart monit (there are many ways, @ahrasis has a commonly referenced howto/script which could be modified to do it, or I have posted some which check the certificate in use and restart services if needed, or you could probably just create a little certbot hook which does it right when the certificate is updated).


To be continued.... (can't put this all in one post here, 10k character limit)

 

The last piece (unless I overlooked something is the apache setup. I'll just cut & paste that section from the tutorial document and try to clean up formatting, though sometimes things don't just paste from one wysiwyg to the next, so if something seems weird/wrong just holler and I can paste from the original source files, etc.:

Proxy
We will setup a proxy connection for /monit to the local monit server.

Disable forward proxies
First secure the proxy config so forward proxies won't run; create /etc/apache2/conf-available/proxyrequests.conf with:

Code:

<IfModule mod_proxy.c>
    ProxyRequests Off
</IfModule>

/monit config
Now the /monit proxy configuration. There is a nice config for a /monit proxy in a virtual host at https://www.ask-sheldon.com/setup-monit-behind-apache-proxy-ispconfig/ which this is loosely based on, but we change from the default monit username/password authentication to certificate based authentication, and add session checking so that you must be logged into ISPConfig in order to view /monit.

Note this config includes your ispconfig mysql password, which can be found in /usr/local/ispconfig/server/lib/config.inc.php:

Code:

grep -E '^\$conf\[.db_(database|user|password)' /usr/local/ispconfig/server/lib/config.inc.php

Save as /etc/apache2/conf-available/ispconfig-monit.conf and change YOUR_DB_PASSWORD:

Code:

# conf-available/ispconfig-monit.conf: configures /monit proxy to local monit server.
#
# this file is included by the ISPConfig vhost,
# it is not intended to be enabled globally (do not a2enconf this file)

<IfModule mod_dbd.c>
  <IfModule mod_authz_dbd.c>

    # mod_dbd configuration
    DBDriver mysql
    DBDParams "dbname=dbispconfig user=ispconfig pass=YOUR_DB_PASSWORD"

    DBDMin 4
    DBDKeep 8
    DBDMax 20
    DBDExptime 300

    <IfModule mod_setenvif.c>
      SetEnvIf Cookie "PHPSESSID=([^ ;]+)" phpsessid=$1
    </IfModule>

    <IfModule mod_proxy_http.c>
      ProxyMaxForwards 5

      SSLProxyEngine On
      SSLProxyMachineCertificateFile "/etc/ssl/private/apache-proxymachine.pem"
      SSLProxyCACertificateFile "/etc/ssl/private/apache-proxymachine.pem"

      <Location /monit>
        ProxyPass "https://127.0.0.1:2812"
        ProxyPassReverse "https://127.0.0.1:2812"
        ProxyPreserveHost On
        ProxyPassReverseCookiePath "/" "/monit/"

        <IfModule mod_auth_env.c>
          AuthType Env
          AuthEnvUser phpsessid
        </IfModule>

        <RequireAll>
          Require env phpsessid
          Require dbd-group monit
        </RequireAll>

        AuthzDBDQuery "SELECT 'monit' FROM sys_session WHERE session_id = %s and session_data like '%%monitor/show_monit.php%%'"

        <IfModule mod_headers.c>
          Header set Cache-Control "no-cache, no-store, max-age=0, must-revalidate"
          Header set Pragma no-cache
          Header set Expires 0
        </IfModule>

      </Location>
    </IfModule>

  </IfModule>
</IfModule>

mod_auth_env
This configuration uses authz_dbd to check the session to see if it is currently logged into ISPConfig and allowed to view monit, and it requires mod_auth_env, which is not currently available as a debian package, so download, compile and install it:

Code:

apt-get install apache2-dev
my_tmp=`mktemp -d`
pushd $my_tmp
wget https://github.com/marctjones/mod_auth_env/archive/master.zip
unzip master.zip 
cd mod_auth_env-master/
debian/rules binary
cd ..
dpkg -i libapache2-mod-auth-env*.deb 
popd
rm -r $my_tmp

ProxyMachine certificate
The proxy config also specifies a certificate (/etc/ssl/private/apache-proxymachine.pem) used to authenticate apache to monit, so we'll need to generate that. Save as /usr/local/sbin/apache-proxymachine-certificate-generater.sh:

Code:

#!/bin/bash

umask 077

APACHE_PEM="/etc/ssl/private/apache-proxymachine.pem"
MONIT_CLIENTPEM="/etc/ssl/private/monit-clientpemfile.pem"
LE_CERT="/etc/letsencrypt/live/`hostname -f`/cert.pem"
LE_CHAIN="/etc/letsencrypt/live/`hostname -f`/chain.pem"
LE_FULLCHAIN="/etc/letsencrypt/live/`hostname -f`/fullchain.pem"
LE_KEY="/etc/letsencrypt/live/`hostname -f`/privkey.pem"
CERTS="/etc/ssl/certs/"

TMP_CRT="`mktemp ${APACHE_PEM}-crt.XXXXXXX`"
TMP_KEY="`mktemp ${APACHE_PEM}-key.XXXXXXX`"

if [ ! -f ${TMP_KEY} -o ! -f ${TMP_CRT} ]
then
  echo "unable to create tmp files to generate new key/certificate" 1>&2
  exit 1
fi

O=`hostname -d`
CN=`hostname -f`
SUBJ="/O=${O}/OU=Monit/CN=${CN}"

openssl req -x509 -days 400 -nodes -newkey rsa:2048 -keyout ${TMP_KEY} -out ${TMP_CRT} -subj ${SUBJ} 2>/dev/null

# Apache needs rsa key then crt
openssl rsa -in ${TMP_KEY} -out ${APACHE_PEM} 2>/dev/null
cat ${TMP_CRT} >> ${APACHE_PEM}

# Monit clientpemfile needs crt for apache and crt chain for monit cli
if [ -f ${LE_KEY} -a -f ${LE_FULLCHAIN} ]
then
  cat ${LE_KEY} ${LE_FULLCHAIN} > ${MONIT_CLIENTPEM}
fi

function get_issuer_certs() {
  for F in ${@}
  do 
    if [ -f ${F} ]; then
      echo -n "${F} "
      F_HASH=`openssl x509 -hash -noout -in ${F}`
      F_ISSUER_HASH=`openssl x509 -issuer_hash -noout -in ${F}`
      if [ "${F_HASH}" != "${F_ISSUER_HASH}" ]; then
        get_issuer_certs ${F_ISSUER_HASH}
      fi
    elif [ -f ${CERTS}/${F}.0 ]; then
      get_issuer_certs ${CERTS}/${F}.*
    else
      echo "get_issuer_certs: unknown input: ${F}" 1>&2
    fi
  done
}

if [ -f ${LE_CHAIN} ]
then
  cat `get_issuer_certs ${LE_CHAIN}` >> ${MONIT_CLIENTPEM}
fi

cat ${APACHE_PEM} >> ${MONIT_CLIENTPEM}

chown root:root ${APACHE_PEM} ${MONIT_CLIENTPEM}
chmod 600 ${APACHE_PEM} ${MONIT_CLIENTPEM}
rm ${TMP_CRT} ${TMP_KEY}

Enable and reload
Now set file permissions, enable our apache config and modules, and run that script to generate the certificate:

Code:

chown root:root /etc/apache2/conf-available/ispconfig-monit.conf
chmod 640 /etc/apache2/conf-available/ispconfig-monit.conf

chmod +x /usr/local/sbin/apache-proxymachine-certificate-generater.sh
/usr/local/sbin/apache-proxymachine-certificate-generater.sh

apt-get install libaprutil1-dbd-mysql

a2enconf proxyrequests
a2enmod proxy_http authz_dbd auth_env
# note we do NOT a2enconf ispconfig-monit

service apache2 restart
service monit restart

conf-custom file
One of the first things we did in this tutorial was modify the ispconfig vhost file to include /etc/apache2/conf-available/ispconfig-*.conf files, but that needs to be made permanent:

Code:

wget -O /usr/local/ispconfig/server/conf-custom/apache_ispconfig.vhost.master https://git.ispconfig.org/ispconfig/ispconfig3/raw/stable-3.1/install/tpl/apache_ispconfig.vhost.master

sed -i -e "/ServerAdmin/a\\ 
  ServerName `hostname -f`\\
  IncludeOptional conf-available/ispconfig-*.conf" /usr/local/ispconfig/server/conf-custom/apache_ispconfig.vhost.master

ISPConfig configuration
Now apache and monit should be ready, the only thing remaining is to set the monit url in ISPConfig. Under System > Server Config > {your.server.name} set the Monit URL to: https://[SERVERNAME]/monit/ (leave the literal [SERVERNAME] placeholder). The Monit user and password should be empty.

That should be everything, wait a couple minutes since monit was just restarted, then head over to Monitor > Monit and it should be working.



Note from the original tutorial you would have already run something like this, but is missing here:

Code:

sed -i -e "/ServerAdmin/a\\ 
  ServerName `hostname -f`\\
  IncludeOptional conf-available/ispconfig-*.conf" /etc/apache2/sites-available/ispconfig.vhost

service apache2 restart

 

No, but we are aware of the issue and looking into the best fix - there is some code that might work, otherwise, we will probably remove the function from ISPConfig.
See https://git.ispconfig.org/ispconfig/ispconfig3/-/issues/4688 aswell.

 

I've added your note to the issue on gitlab