letsencrypt breaks SSL, fails to issue, ispconfig 3.1.13

Discussion in 'ISPConfig 3 Priority Support' started by ronee, Feb 1, 2019.

Tags:
  1. ronee

    ronee Member HowtoForge Supporter

    We've been using ispconfig with letsencrypt without issue on quite a few servers, this one however we're having difficulty with. (OS is debian 9)

    - All site hostnames and aliases properly resolve
    - certbot is installed:

    # dpkg -l | grep certbot
    ii certbot 0.28.0-1~deb9u1 all automatically configure HTTPS using Let's Encrypt
    ii python-certbot 0.10.2-1 all main library for certbot
    ii python3-certbot 0.28.0-1~deb9u1 all main library for certbot

    When letsencrypt checkbox is enabled for a given site, the checkbox does not stay checked however the vhost file is changed and the references to the cert and key files are removed from the file. The result of this is that the server responds with http traffic on port 443, so https is not possible but urls like this work: http://host.domain:443/robots.txt

    The letsencrypt.log contains only this:

    2019-01-31 11:00:06,544:INFO:certbot.main:Saving debug log to /var/log/letsencrypt/letsencrypt.log
    2019-01-31 11:00:06,544:DEBUG:certbot.main:certbot version: 0.10.2
    2019-01-31 11:00:06,544:DEBUG:certbot.main:Arguments: ['-n', '--post-hook', "echo '1' > /usr/local/ispconfig/server/le.restart"]
    2019-01-31 11:00:06,545:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#webroot,PluginEntryPoint#null,PluginEntryPoint#manual,PluginEntryPoint#standalone)
    2019-01-31 11:00:06,545:DEBUG:certbot.renewal:no renewal failures

    The only file present in /etc/letsencrypt is cli.ini

    This server is behind a TCP load balancer / firewall and ports 80 & 443 are natted to the web server. The server's hostname does resolve using local DNS to the web server's local IP. The sites of course resolve to the public IP which is assigned to the external interface of the firewall/load balancer.

    Would be interested in recommendations on troubleshooting this.

    Thanks in advance
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    There is really nothing else in /etc/letsencrypt/ folder then cli.ini? Here the ls output from one of my servers:

    ls -la /etc/letsencrypt/
    total 40
    drwxr-xr-x 9 root root 4096 Feb 1 03:01 .
    drwxr-xr-x 103 root root 4096 Jan 30 11:11 ..
    drwx------ 6 root root 4096 Feb 1 01:05 accounts
    drwx------ 19 root root 4096 Jan 10 17:18 archive
    drwxr-xr-x 2 root root 4096 Feb 1 01:06 csr
    drwx------ 2 root root 4096 Feb 1 01:06 keys
    drwx------ 19 root root 4096 Jan 10 17:18 live
    -rw-r--r-- 1 root root 1389 Apr 7 2016 options-ssl-apache.conf
    drwxr-xr-x 2 root root 4096 Feb 1 01:06 renewal
    drwxr-xr-x 5 root root 4096 Feb 19 2018 renewal-hooks

    If there are no such folders on your server, then probably the folder was accidentally wiped out. Do you have a backup of that folder that you can restore?
     
  3. ronee

    ronee Member HowtoForge Supporter

    strange, I know of nothing that would have wiped out that directory, this is a newly installed server, I have checked another server and I have directories similar to what you listed but the server in question has only the one file, this is not a directory we normally back up - should we?

    Could I perhaps copy some of the files from another server or reinstall some component to recreate what is needed?

    thanks
     
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Which tutorial did you use? Probably certbot was not initiated correctly. I recently updated the Debian perfect server guides. maybe I missed something there.
     
  5. ronee

    ronee Member HowtoForge Supporter

    This server is based on a virtual template from which we've installed a great many servers. Basically the prerequisites are all set and we just set the IP address, ssh keys, hostnames and root passwords and then install the latest stable ispconfig. Then if we plan to use letsencrypt we install certbot.
    In troubleshooting this we did update certbot - there was a recent change there due to letsencrypt terminating TLS-SNI-01, perhaps that is a factor? There is really nothing different or unusual with this one.

    We've installed a great many servers from this template and this is the first time we've had an issue like this.
    There are developers actively working with this server so I'd rather see if there's a way to fix the problem rather than going to a new install.
    Anything that can be done? remove and reinstall certbot perhaps?

    Thanks
     
  6. ronee

    ronee Member HowtoForge Supporter

    We did install certbot using apt-get, I see the current perfect server docs say to wget and install it manually, perhaps that makes a difference?
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    Installing it with apt should be fine, that's what the tutorials used before. I just changed it as there was no certbot update which was required for a change at LE in February, but this was fixed by Debian so my change to install it manually is not needed anymore. Maybe you should try to uninstall certbot and then reinstall it. If this don't help, try the method which is used in the tutorials now.
     
  8. ronee

    ronee Member HowtoForge Supporter

    Thanks Till, good news somehow I fixed it. I removed using apt-get remove and then installed manually as listed currently in the perfect server docs. At first run I had the same issue where the self-signed cert references were removed and the server began once again responding with http on port 443 so I reverted once again to a self-signed (created new in the SSL tab).

    I then found this thread on the letsencrypt forums that seemed relevant:
    https://github.com/certbot/certbot/issues/4890

    I then grepped through all the config files in /etc/apache2 and found no references to /etc/letsencrypt, then I noticed that in addition to cli.ini there was another subdirectory present - renewal-hooks, so I tried re-enabling letsencrypt once more and then all worked as it usually does.

    My theory is that somehow there was a reference to letsencrypt in one of the config files which during the steps I took were deleted. Perhaps there was even a vhost.err file that certbot parsed and decided to exit without taking action.

    Hopefully this troubleshooting helps someone else.

    Thanks as always for your support.

    Best,
    Ron
     

Share This Page