We recently added a mirror server as a backup to our existing ispconfig web and mail server (ubuntu 16.04 nginx php-fpm ispconfig3 - hosting mutliple websites and cms). We shared the necessary folders (the master machine running a NFS server sharing /var/www and the /etc/letsencrypt/ and /usr/local/ispconfig/interface/acme). Soon after activating the mirror we noticed missing LE checkboxes for all the sites in ispc. After activating the option "Let's Encrypt SSL" for a site again the letsencrypt.log shows a successful renewal attempt, but soon after we receive a mail with "WARNING - Let's Encrypt SSL Cert for: domain.tld could not be issued.". Those mails seem to always be sent from the mirror server. The certs are ok and keep working fortunately but the checkbox gets unchecked and we have to take care not to save a site with the box uncheck to avoid loosing TLS. It is the same when creating a new cert the log shows no problems, we get the new working cert and all looks fine besides the fact that after a minute the checkbox is unchecked. Please advise how to deal with this situation. All we need from this mirror server is to act as a backup slave - we have deactivated the ../server/lib/classes/conf.d/500-backup skript on the master by renaming. Maybe this has an effect? We do not understand why there is a feedback coming from the mirror that most likely causes the option to get unchecked at the panel. The master was running fine for quite a while until the mirror was added and nothing else was changed. We ridded letsencrypt from entries that caused errors to show up within the logs of both servers, to be sure that there is no general reaction because of a failing exit code from letsencrypt or something like that. We have ntp time synchronization via ubuntu ntp server package on both servers. Version on both servers identically letsencrypt 0.4.1 Last but not least, thanks a lot for maintaining this great product and for beeing active in the forum!
This might be a race condition.I guess we might have to add a test in the code to create the LE cert on the master only and maybe add a delay to creating the vhost on the slave.
Thank you very much for the quick answer, you guys are really doing a great job! I was wondering if it would be a solution in our case to set the mirror inactive, since i really only want it to run the backups - could this work? If I understand it correctly we share the LE folders to provide that it does not matter which server is sending the initial request to LE - is it the case that they both have to verify that this was successful? I understand that ispc is desigend in a way that a mirror server is always running all the services that the master is running - which is ofc necessary in the case that the mirror has to take over when master has a problem. Maybe we can have some kind of passive mirror mode added to ispc?
Yes, that's the reason why the folders are shared. ISPConfig checks if a cert exists, so the slave should not try to create the cert again. and I guess that's where we run into an issue at the moment, so either we have to add a delay of e.g. 10 seconds on the slave to ensure that the master fetched the ssl cert successfully or we disable LE completely on the slaves so only the master fetches and handles the LE certs. Basically, the mirror should work as active and passive mirror, just the LE function does not work well in passive mode yet. I'll add this to the bug tracker and we'll add a fix for it.
Well, the question is what happens if the mirror has to take over. But some kind of passive mirror setting would be something to think about. I am still wondering what happens if we set the mirror server as inactive in ispc - I guess the setting will lead to server.sh not beeing run anymore - is that correct? Which would mean the backup script will not run anymore as well I am afraid. The reason I had to add this mirror is that backups on this server take too much time and impact the webserver performance, this was nicely taken care of by running backups only on the mirror. Right now I have to live with the fact that we have to check for the certs manually since they won´t be renewed automatically if the box is not checked, or is this not the case maybe? I would be glad to get some assertion for this situation.
+1 same here NOTICE: You have to "reenable" LE on main web after every new subdomain or alias for web otherwise ISPC dont add it to LE request.
Problem like such has as well surfaced on multi server setups...assertions to this would be welcoming and appreciated...
