LetsEncrypt on Debian Jessie + ISP3.12 - automatic fails, manually succeed

Discussion in 'Installation/Configuration' started by muelli75, Mar 18, 2017.

  1. muelli75

    muelli75 Member

    Hi out there!

    Im working with ISPConfig 3.12 on Jessie. ISPconfig do its job, but LetsEncrypt isnt working by klicking on the "Lets Encrypt SSL"-Button in the Web Domain-Panel.

    The ISPConfig-Log shows warnings like "Let's Encrypt SSL Cert for: FQDN.at could not be issued." and "Could not verify domain www.FQDN.at, so excluding it from letsencrypt request."

    This warnings are sadly NOT written to /var/log/letsencrypt/letsencrypt.log

    If I try a manual installation of a LE-Certificate by
    certbot certonly --webroot -w /var/www/FQDN.at/web/ -d FQDN.at -d www.FQDN.at​
    the verification fails (log is anonymized)

    Failed authorization procedure. www.FQDN.at (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.FQDN.at/.well-known/acme-challenge/dZsMOMyBEg8hOqcnoH2UvpgzEACybcA_m6h_njp2eQE: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"

    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

    <ht", FQDN.at (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://kunststoffschilder.at/.well-known/acme-challenge/UioFNCfZR2DEcnKspuM-b3pTODf1I7SKeYyydFPlIsE: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <ht"

    IMPORTANT NOTES:
    - The following errors were reported by the server:
    Domain: www.FQDN.at
    Type: unauthorized
    Detail: Invalid response from
    http://www.FQDN.at/.well-known/acme-challenge/dZsMOMyBEg8hOqcnoH2UvpgzEACybcA_m6h_njp2eQE:
    "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <ht"​

    If I do a manual installation of a LE-Certificate in standalone-mode
    certbot certonly --standalone -d FQDN.at -d www.FQDN.at​
    the certifiacte is created without errors.

    I checked if I can reach in a browser
    and I do. As result I get the error "forbidden" (403).

    For some reasons the log-files /var/log/letsencrypt/letsencrypt.log are only written if my manual request fails or succeeded.

    Are there any ideas how to solve this issue?

    Thanks for reading and replys!

    Martin
     
    muelli75, Mar 18, 2017
    #1
  2. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    How did you install your LE/certbot at the first place? Was it before or after installing/updating ISPC?
     
    ahrasis, Mar 19, 2017
    #2
  3. florian030

    florian030 Well-Known Member HowtoForge Supporter

    ISPConfig tries to connect to each site before getting a cert for the domain. If your could not connect to the domain, this domain will be excluded. It seems, that this not working very well on all systems (we already discussed this).
    I disabled this check on my servers. new:
    Code:
      $le_domains = array();
  foreach($temp_domains as $temp_domain) {
//  $le_hash_check = trim(@file_get_contents('http://' . $temp_domain . '/.well-known/acme-challenge/' . $le_rnd_file));
//  if($le_hash_check == $le_rnd_hash) {
  $le_domains[] = $temp_domain;
  $app->log("Verified domain " . $temp_domain . " should be reachable for letsencrypt.", LOGLEVEL_DEBUG);
//  } else {
//  $app->log("Could not verify domain " . $temp_domain . ", so excluding it from letsencrypt request.", LOGLEVEL_WARN);
//  }
  }
    old:
    Code:
      $le_domains = array();
  foreach($temp_domains as $temp_domain) {
    $le_hash_check = trim(@file_get_contents('http://' . $temp_domain . '/.well-known/acme-challenge/' . $le_rnd_file));
    if($le_hash_check == $le_rnd_hash) {
  $le_domains[] = $temp_domain;
  $app->log("Verified domain " . $temp_domain . " should be reachable for letsencrypt.", LOGLEVEL_DEBUG);
    } else {
    $app->log("Could not verify domain " . $temp_domain . ", so excluding it from letsencrypt request.", LOGLEVEL_WARN);
    }
  }
    lines 1300+ in nginx_plugin.inc.php and lines 1227+ in apache2_plugin.inc.php
     
    florian030, Mar 19, 2017
    #3
  4. muelli75

    muelli75 Member

    Hi!

    Thanks for reply on this thread! I installed LE/certbot after installing ISPconfig.

    @florian030 hints work like a charm. I was able to request a certificate for a webspace/domain by clicking on "Lets Encrypt SSL"-Button in the Web Domain-Panel.

    But ...

    Surfing with Firefox to https://www.FQDN1.at results in an SSL-Error.

    Your connection is not secure

    The owner of www.geruest-FQDN1.at has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website.

    Learn more…

    Report errors like this to help Mozilla identify and block malicious sites

    www.geruest-FQDN1.at uses an invalid security certificate.

    The certificate is only valid for the following names:
    FQDN2.at, www.FQDN2.at

    Error code: SSL_ERROR_BAD_CERT_DOMAIN
    ​
    I requested a manual certificate (standalone) for FQDN2 which was successfull installed.

    Any ideas to solve to this? Thanks for answers!

    Regards, Martin
     
    muelli75, Mar 19, 2017
    #4
  5. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Check your website vhost files for LE SSL links i.e. whether they exist and are pointing to the right place.
     
    ahrasis, Mar 19, 2017
    #5
  6. muelli75

    muelli75 Member

    Thanks ahrasis for the tipp!

    The ssl-confs for all the other vhosts (vhost1, vhost2, vhost3, etc) are alle pointing to another vhosts ssl-directory. All ssl-direvtives pointing to

    SSLCertificateFile /etc/letsencrypt/live/www.FQDN3.at/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/www.FQDN3.at/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
    ​
    After pointing to the correct location all works well.

    Is this a common error by using LE in the ISPC-Panel? Do you know how comes this issue?

    Best regards, Martin
     
    muelli75, Mar 25, 2017
    #6
  7. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    No. I don't think it is common. Do you by chance have a custom vhost template?
     
    ahrasis, Mar 25, 2017
    #7
  8. muelli75

    muelli75 Member

    Hi out there!

    Today I added a new webspace - and I tried do install a LE-certificate. But I got a well known error:

    "url": "http://www.FQDN4.at/.well-known/acme-challenge/58AIIkphCFO9FAmzirC44JEQ3fD8DOEoS6_pRS4Vrxs",\n "hostname": "www.FQDN4.at",\n "port": "80",\n "addressesResolved": [\n "136.243.XXX.172"\n ],\n "addressUsed": "136.243.XXX.172"\n }\n ]\n }\n ],\n "combinations": [\n [\n 0\n ],\n [\n 1\n ],\n [\n 2\n ]\n ]\n}'
    2017-04-09 16:53:13,094:DEBUG:certbot.reporter:Reporting to user: The following errors were reported by the server:

    Domain: www.FQDN4.at
    Type: unauthorized
    Detail: Invalid response from http://www.FQDN4.at/.well-known/acme-challenge/58AIIkphCFO9FAmzirC44JEQ3fD8DOEoS6_pRS4Vrxs: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <ht"

    Domain: FQDN4.at
    Type: unauthorized
    Detail: Invalid response from http://FQDN4.at/.well-known/acme-challenge/37070pfTyaaVD0pFxRdKrwJwPcL-FtDlwAB70aJvXNg: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    <ht"

    To fix these errors, please make sure that your domain name was entered correctly and the DNS A record(s) for that domain contain(s) the right IP address.
    2017-04-09 16:53:13,094:INFO:certbot.auth_handler:Cleaning up challenges
    2017-04-09 16:53:13,094:DEBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/37070pfTyaaVD0pFxRdKrwJwPcL-FtDlwAB70aJvXNg
    2017-04-09 16:53:13,094:DEBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/58AIIkphCFO9FAmzirC44JEQ3fD8DOEoS6_pRS4Vrxs
    2017-04-09 16:53:13,094:INFO:certbot.plugins.webroot:Unable to clean up challenge directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge
    2017-04-09 16:53:13,094:DEBUG:certbot.plugins.webroot:Error was: [Errno 39] Directory not empty: '/usr/local/ispconfig/interface/acme/.well-known/acme-challenge'
    2017-04-09 16:53:13,095:DEBUG:certbot.main:Exiting abnormally:
    Traceback (most recent call last):
    File "/usr/bin/letsencrypt", line 9, in <module>
    load_entry_point('certbot==0.9.3', 'console_scripts', 'certbot')()
    File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 776, in main
    return config.func(config, plugins)
    File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 563, in obtain_cert
    action, _ = _auth_from_domains(le_client, config, domains, lineage)
    File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 100, in _auth_from_domains
    lineage = le_client.obtain_and_enroll_certificate(domains)
    File "/usr/lib/python2.7/dist-packages/certbot/client.py", line 281, in obtain_and_enroll_certificate
    certr, chain, key, _ = self.obtain_certificate(domains)
    File "/usr/lib/python2.7/dist-packages/certbot/client.py", line 253, in obtain_certificate
    self.config.allow_subset_of_names)
    File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 78, in get_authorizations
    self._respond(resp, best_effort)
    File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 135, in _respond
    self._poll_challenges(chall_update, best_effort)
    File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 199, in _poll_challenges
    raise errors.FailedChallenges(all_failed_achalls)
    FailedChallenges: Failed authorization procedure. www.FQDN4.at (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://www.FQDN4.at/.well-known/acme-challenge/58AIIkphCFO9FAmzirC44JEQ3fD8DOEoS6_pRS4Vrxs: "<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    ​
    The domain is few days old, the DNS-entrys are pointing to the right server. The webspace is reachable in a browser by typing the FQDN.

    Searching syslog shows following:
    Apr 9 19:12:09 monarch apache2[14753]: Starting web server: apache2[Sun Apr 09 19:12:09.511687 2017] [alias:warn] [pid 14768] AH00671: The Alias directive in /etc/apache2/conf-enabled/letsencrypt.sh.conf at line 18 will probably never match because it overlaps an earlier Alias.
    Apr 9 19:12:09 monarch apache2[14753]: AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:69

    And letsencrypt.sh.conf (line 16 to 19) shows:
    <IfModule alias_module>
    # Serve ACME challenge responses
    Alias /.well-known/acme-challenge/ /var/lib/letsencrypt.sh/acme-challenges/
    </IfModule>​

    Maybe thats a fault? Any further ideas?

    Thanks for all inputs!

    Best regards, Martin
     
    Last edited: Apr 9, 2017
    muelli75, Apr 9, 2017
    #8
  9. muelli75

    muelli75 Member

    Solved! Finally (i hope).

    Its a little confusing having two scripts which are doing the same, but under different names.
    letsencrypt.sh is renamed to dehydrated because of avoiding trademark-problems.
    https://abdussamad.com/archives/1162-Lets-Encrypt-Dehydrated-on-Debian-Jessie.html
    https://www.aaflalo.me/2016/09/dehydrated-bash-client-lets-encrypt/

    I did all the changes in the config-files related to letsencrypt.sh instead of dehydrated.
    So changing the alias and directory-directive in /etc/apache2/conf-available/dehydrated.conf to /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/ did it.

    All certificates are issued well by clicking on "LetsEncrypt" in ISPC3.1.2 -> Websites/Webdomain

    Thanks for all answers to this thread!
     
    muelli75, Apr 15, 2017
    #9
    ahrasis likes this.
  10. zenny

    zenny Member

    It would be nice if you share the configs of dehydrated/letsencrypt.sh to adapt to the ispconfig3 with nginx server! I could not find anything nginx-equivalent to /etc/apache2/conf-available/dehydrated.conf.

    BTW, I am on ISPConfig 3.1.5 and all DNS settings are working fine, but failed to renew with error that reads:
    "Could not verify domain <DOMAINNAME>, so excluding it from letsencrypt request."
    where letsencrypt for all <DOMAINNAME>s have been issued earlier and was not renewing automatically.
    Thanks!
     
    zenny, Jul 12, 2017
    #10
  11. sjau

    sjau Local Meanie Moderator

    @zenny: If you host the dns for your domains with ISPConfig, you migh want to have a look at the acme.sh le-client. It has automated DNS-01 authorization for ISPConfig support. Basically it means, you get certs by adding a DNS TXT record to the DNS zone file of the domain. So it's independant of any webserver running.
     
    sjau, Jul 13, 2017
    #11
  12. muelli75

    muelli75 Member

    Hi zenny!

    Lets have a look at /etc/apache2/conf-available/dehydrated.conf (remember thats for apache 2.4)

    # Apache configuration to serve http-01 ACME challenges responses.
    # This is included from the dehydrated-apache2 package, thought to be used
    # with dehydrated as packaged in Debian.

    <IfModule proxy_module>
    # Do not proxy ACME challenge responses
    ProxyPass /.well-known/acme-challenge/ !
    </IfModule>

    <IfModule !alias_module>
    # Load the alias module, if not loaded already
    Include /etc/apache2/mods-available/alias.load
    Include /etc/apache2/mods-available/alias.conf
    </IfModule>

    <IfModule alias_module>
    # Serve ACME challenge responses
    #Alias /.well-known/acme-challenge/ /var/lib/dehydrated/acme-challenges/
    Alias /.well-known/acme-challenge/ /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/
    </IfModule>

    #<Directory /var/lib/dehydrated/acme-challenges/>
    <Directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/>
    Options FollowSymlinks
    Options -Indexes
    AllowOverride None
    # Apache >= 2.3
    <IfModule mod_authz_core.c>
    Require all granted
    </IfModule>
    # Apache < 2.3
    <IfModule !mod_authz_core.c>
    Order Allow,Deny
    Allow from all
    </IfModule>
    </Directory>
    ​
    HTH

    Best regards, Martin
     
    muelli75, Jul 13, 2017
    #12
  13. manyk

    manyk New Member

    If you talk about a new server (ISPConfig slave) that you configured and added to the cluster then you shuld also check the /etc/hosts files.
    Some distributions or install scripts tend to add your server FQDN in that file paired with 127.0.0.1.

    This created problems when you try to access the website from the same machine beacause the webserver does not know how to server that website on a localhost request.
     
    manyk, Sep 20, 2017
    #13

Share This Page