ISPConfig 3.1 + Letsencrypt

Discussion in 'General' started by matthewobrn, Aug 27, 2016.

  1. matthewobrn

    matthewobrn New Member

    I appear to be having a couple of issues with my ISPconfig and letsencrypt configuration.
    The question 1st I have is, can I create a subdomain in ISPconfig such as and only have a cname dns record for * to and an A record for to my ip address. Will this be sufficient validation for letencrypt or do I need an A record for each subdomain I want to use? I tried setting up my own dns server but turns out my isp is blocking port 53 so had to bail on that idea.

    The 2nd question or problem is a little more complicated, I best explain my setup first.
    I have 3 domains registered.
    all of them have dns records as follows:
    Code: A          3600
    *          CNAME      3600 MX         3600 10 NS         3600 NS         3600 NS         3600 
    When I create a website in ISPconfig for with a ssl cert from letsencrypt it works.
    Then I create a site for also with a ssl cert from letsencrypt and when I try to view the page is serves the ssl cert for
    The same for
    If I create a subdomain as a website such as letsencrypt fails.
    Here is the letsencrypt log for
    2016-08-27 03:49:14,521:DEBUG:certbot.main:Root logging level set at 30
    2016-08-27 03:49:14,525:INFO:certbot.main:Saving debug log to /var/log/letsencrypt/letsencrypt.log
    2016-08-27 03:49:14,528:WARNING:certbot.cli:You are running with an old copy of letsencrypt-auto that does not receive updates, and is less reliable than more recent versions. We recommend upgrading to the latest certbot-auto script, or using native OS packages.
    2016-08-27 03:49:14,529:DEBUG:certbot.main:certbot version: 0.8.1
    2016-08-27 03:49:14,529:DEBUG:certbot.main:Arguments: ['-n', '--text', '--agree-tos', '--expand', '--authenticator', 'webroot', '--server', '', '--rsa-key-size', '4096', '--email', '[email protected]', '--domains', '', '--webroot-path', '/usr/local/ispconfig/interface/acme']
    2016-08-27 03:49:14,533:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#apache,PluginEntryPoint#webroot,PluginEntryPoint#null,PluginEntryPoint#manual,PluginEntryPoint#standalone)
    2016-08-27 03:49:14,535:DEBUG:certbot.plugins.selection:Requested authenticator webroot and installer None
    2016-08-27 03:49:14,555:DEBUG:certbot.plugins.selection:Single candidate plugin: * webroot
    Description: Place files in webroot directory
    Interfaces: IAuthenticator, IPlugin
    Entry point: webroot = certbot.plugins.webroot:Authenticator
    Initialized: <certbot.plugins.webroot.Authenticator object at 0xb5fa6910>
    Prep: True
    2016-08-27 03:49:14,559:DEBUG:certbot.plugins.selection:Selected authenticator <certbot.plugins.webroot.Authenticator object at 0xb5fa6910> and installer None
    2016-08-27 03:49:15,944:DEBUG:certbot.main:Picked account: <Account(8673582473deb8f96d8dff55f84f5522)>
    2016-08-27 03:49:15,954:DEBUG:root:Sending GET request to args: (), kwargs: {}
    2016-08-27 03:49:15,972:INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1):
    2016-08-27 03:49:18,296:DEBUG:requests.packages.urllib3.connectionpool:"GET /directory HTTP/1.1" 200 280
    2016-08-27 03:49:18,302:DEBUG:root:Received <Response [200]>. Headers: {'Content-Length': '280', 'Expires': 'Sat, 27 Aug 2016 03:49:14 GMT', 'Boulder-Request-Id': 'oXlxU0zagJqnOvVTqofeNM2_IRayhsUknH-nJt0Szd4', 'Strict-Transport-Security': 'max-age=604800', 'Server': 'nginx', 'Connection': 'keep-alive', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Sat, 27 Aug 2016 03:49:14 GMT', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json', 'Replay-Nonce': 'R3jp8PPQbneE6K7jXmHn0PX3bdTA-A7eRqQ3-WShVfA'}. Content: '{\n  "new-authz": "",\n  "new-cert": "",\n  "new-reg": "",\n  "revoke-cert": ""\n}'
    2016-08-27 03:49:18,304:DEBUG:acme.client:Received response <Response [200]> (headers: {'Content-Length': '280', 'Expires': 'Sat, 27 Aug 2016 03:49:14 GMT', 'Boulder-Request-Id': 'oXlxU0zagJqnOvVTqofeNM2_IRayhsUknH-nJt0Szd4', 'Strict-Transport-Security': 'max-age=604800', 'Server': 'nginx', 'Connection': 'keep-alive', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Sat, 27 Aug 2016 03:49:14 GMT', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json', 'Replay-Nonce': 'R3jp8PPQbneE6K7jXmHn0PX3bdTA-A7eRqQ3-WShVfA'}): '{\n  "new-authz": "",\n  "new-cert": "",\n  "new-reg": "",\n  "revoke-cert": ""\n}'
    2016-08-27 03:49:18,312:WARNING:certbot.main:Renewal conf file /etc/letsencrypt/renewal/ is broken. Skipping.
    2016-08-27 03:49:18,328:DEBUG:certbot.main:Traceback was:
    Traceback (most recent call last):
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 237, in _find_duplicative_certs
        candidate_lineage = storage.RenewableCert(renewal_file, cli_config)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 242, in __init__
        "file reference".format(self.configfile))
    CertStorageError: renewal config file {} is missing a required file reference
    2016-08-27 03:49:18,331:WARNING:certbot.main:Renewal conf file /etc/letsencrypt/renewal/ is broken. Skipping.
    2016-08-27 03:49:18,333:DEBUG:certbot.main:Traceback was:
    Traceback (most recent call last):
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 237, in _find_duplicative_certs
        candidate_lineage = storage.RenewableCert(renewal_file, cli_config)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 242, in __init__
        "file reference".format(self.configfile))
    CertStorageError: renewal config file {} is missing a required file reference
    2016-08-27 03:49:18,353:DEBUG:root:Requesting fresh nonce
    2016-08-27 03:49:24,314:INFO:certbot.reporter:Reporting to user: The following errors were reported by the server:
    Type:   unauthorized
    Detail: Invalid response from "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    To fix these errors, please make sure that your domain name was entered correctly and the DNS A record(s) for that domain contain(s) the right IP address.
    2016-08-27 03:49:24,316:INFO:certbot.auth_handler:Cleaning up challenges
    2016-08-27 03:49:24,317:DEBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/GXRXGqdPHRddC2Ri_8WCKPaxYMncXl0hXxBls_ieHkM
    2016-08-27 03:49:24,320:INFO:certbot.plugins.webroot:Unable to clean up challenge directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge
    2016-08-27 03:49:24,321:DEBUG:certbot.plugins.webroot:Error was: [Errno 39] Directory not empty: '/usr/local/ispconfig/interface/acme/.well-known/acme-challenge'
    2016-08-27 03:49:24,336:DEBUG:certbot.main:Exiting abnormally:
    Traceback (most recent call last):
      File "/root/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 744, in main
        return config.func(config, plugins)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 555, in obtain_cert
        _, action = _auth_from_domains(le_client, config, domains, lineage)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 94, in _auth_from_domains
        lineage = le_client.obtain_and_enroll_certificate(domains)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 276, in obtain_and_enroll_certificate
        certr, chain, key, _ = self.obtain_certificate(domains)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 247, in obtain_certificate
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 74, in get_authorizations
        self._respond(resp, best_effort)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 131, in _respond
        self._poll_challenges(chall_update, best_effort)
      File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/", line 195, in _poll_challenges
        raise errors.FailedChallenges(all_failed_achalls)
    FailedChallenges: Failed authorization procedure. (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    I followed the perfect server setup for Debian Jessie and ISPConfig 3.1.

    Thanks for any help.
    Last edited: Aug 27, 2016
  2. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    The concept is sound, but I don't know if letsencrypt specifically will work that way. Get letsencrypt working "normally" then it should be pretty easy to test (and report here, if you would).

    Make sure the SNI checkbox is enabled in ispconfig, and check the IP addr in your vhost settings, make sure they all use either '*" or the ip addr, but don't mix the two or you get some unexpected results.

    Hmm, maybe that answers your first question. Try adding a DNS record for (ie. don't rely on the wildcard dns record) and see if that changes.

    Also in your output there are these, which might be worth looking in to:
  3. matthewobrn

    matthewobrn New Member

    So a little more information. Firstly, I have managed to solve the majority of the issues I was having. I installed another instance of the perfect server on a vmware workstation. I ran all of the same tests on this using the same domain names and urls. I also created the same websites in ISPconfig and tried issuing ssl certs from letsencrypt.
    Everything worked perfectly.
    Baring in mind this was two days after following the same tutorial on my Banana Pi M3. The only main difference is the architecture Arm64 vs Armv71. I did however notice a version difference between the two. Arm64 was ICPConfig 3.1dev and Armv71 was ISPConfig 3.1rc1 (I think). I tried running:
    _____ ___________   _____              __ _
    |_   _/  ___| ___ \ /  __ \            / _(_)
      | | \ `--.| |_/ / | /  \/ ___  _ __ | |_ _  __ _
      | |  `--. \  __/  | |    / _ \| '_ \|  _| |/ _` |
    _| |_/\__/ / |     | \__/\ (_) | | | | | | | (_| |
    \___/\____/\_|      \____/\___/|_| |_|_| |_|\__, |
                                                  __/ |
    >> Update
    Please choose the update method. For production systems select 'stable'.
    WARNING: The update from GIT is only for development systems and may break your current setup. Do not use the GIT version on servers that host any live websites!
    Note: Update all slave server, before you update master server.
    Select update method (stable,git-stable,git-master) [stable]:
    There are no updates available for ISPConfig 3.1rc1
    So I put this down to the different architectures. (my first mistake)
    I began investigating if the letsencrypt url was actually being served to the internet when the authenication scripts were being run. It turns out they were not. I looked at the vhost serving the files "ispconfig.vhost" in the /etc/apache2/sites-avialible/ and noticed a big difference between the ones on my Banana Pi and the vmware workstation.
    The 3.1dev vesion had:
    Alias /.well-known/acme-challenge /usr/local/ispconfig/interface/acme/.well-known/acme-challenge
    <Directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge>
                    Require all granted
    Where as the 3.1rc1 had something like this:
    <IfModule mod_headers.c>
    <LocationMatch "/.well-known/acme-challenge/*">
    Header set Content-Type "text/plain"
    When I changed the files to the newer version it seemed to fix everything. Or so I thought. After I changed some other settings around in the ISPConfig control panel things started going awry again. I eventually decided that it was time for a fresh install of ISPConfig so I downloaded ran the following:
    cd /tmp
    wget -O ISPConfig-3.1-beta.tar.gz
    tar xfz ISPConfig-3.1-beta.tar.gz
    cd ispconfig3-stable-3.1*
    cd install
    php -q uninstall.php
    php -q install.php
    This updated to ISPConfig to 3.1dev on the banana pi and everything works great with regards to the ssl certs and letsencrypt.

    Once the server was configured correctly and all the subdomains had no aliases of there own it worked great. Once I change ISP's and have my own dns running I will set up A records for all the domains. Right now I'm limited to 10 dns record changes per year per domain. (A pain in the ass i know)

    I can't seem to find the sni checkbox in ispconfig. I think it is probably already enabled as its all working.
    Interesting you mention about the '*' or a specific ip address. During my fault finding I found is that in the Apache2 documentation it mentions that there is no difference between:
    <VirtualHost _default_:80>
    <VirtualHost *:80>
    This is not true.
    I had a website setup called and enabled the default alias to be *
    This was so would be directed to the site.
    I couldn't get it working. kept being directed to the default apache index.html at /var/www/html.
    But I did notice the did navigate to
    Investigating the vhost files for 000-default.conf and defaul-ssl.conf the only main difference was the *:80 and _default_:80 once I changed to *:80 in the non ssl file everything worked beautifully.

    I did try adding an A DNS record for one of the domains just as a test with no avail.

    Now onto the next problem, isn't working. I can't find any reference to it in the vhost files or /etc/apache2/conf files. From what I have read there should be a system link to it in the /etc/apache2/conf -> /etc/phpmyadmin/apache.conf

    Can anyone confirm this before I break my server again!
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    You mix up ISPConfig versions here. You tried to downgrade ISPConfig 3.1rc1 to ISPConfig as ISPConfig stable is, so the installer told you that there i no newer version, ISPConfig 3.1 has not been released yet and therefore can not be stable. If you ant to update a beta or RC release, chose git-stable.

    The symlink gets created by the phpmyadmin apt installer. if it is missing, then you did not selected apache2 during install. selecting means moving the cursor to the menu point with [tab] and then activating it with [space]. You can rerun the step with:

    dpkg-reconfigure phpmyadmin
  5. matthewobrn

    matthewobrn New Member

    Cheers for that, should have realised. Something to look out for in the future.

    On a side note, who maintains the tutorial pages? It's only a type in the title, probably not even worth mentioning.
    Step 8
    Step 10
    And for some reason phpMyAdmin must not have finished correctly as the symlink is missing.

    I ran the command and it worked beautifully.
    Everything is now working! Time to do a dd backup onto my sd-card.

Share This Page