Letsencrypt not working anymore

Discussion in 'Installation/Configuration' started by Thomas Schachtner, May 16, 2018.

Tags:
Page 1 of 2
  1. Thomas Schachtner

    Thomas Schachtner New Member

    Hi there,
    I don't exactly since when, but since some weeks, the autmated generation of Letsencrypt certificates via ISPConfig does not work anymore. When a certificate is due to renewal, the following lines are generated by the server.sh script:

    Plugins selected: Authenticator webroot, Installer None
    Obtaining a new certificate
    Performing the following challenges:
    http-01 challenge for test1.example.com
    http-01 challenge for test1.example2.com
    http-01 challenge for test1.example3.com
    Using the webroot path /usr/local/ispconfig/interface/acme for all unmatched domains.
    Waiting for verification...
    Cleaning up challenges
    Unable to clean up challenge directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge
    Failed authorization procedure. test1.example.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http:/ / test1.example.com/.well-known/acme-challenge/y3bguedb9zfaHOssW1Q23he37T4HJ2Eot5ycj7kzPwQ: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>404 Not Found</title>
    </head><body>
    <h1>Not Found</h1>
    <p", test1.example2.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http:/ / test1.example2.com/.well-known/acme-challenge/eY7zN4_TA_FAKm-OkLKigtMBr960xdFZkXD4v7NpS7M: "<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html><head>
    <title>404 Not Found</title>
    </head><body>
    <h1>Not Found</h1>
    <p"
    finished.

    (I've modified the links above to be able to post this message...)
    Does anyone know, why that may occur?
    Is there any way to debug it further?
    Is there any way to get test certificates instead of live certs, as I'm constantly reaching Letsencrypt's rate limits during my tests?

    I'm really stuck at the moment. Any hint would be very welcome.
    I already searched the web, but did not find a solution which fixes this issue on my server.​
     
    Thomas Schachtner, May 16, 2018
    #1
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    To test this, run:

    touch /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/hello.txt

    on the shell to create a hello.txt file. You must be able to reach this then under the url:

    http://test1.example.com/.well-known/acme-challenge/hello.txt

    If not, check if dns for the domain is ok and next, check rewrite rules of that website, maybe you redirect that request to a different location.
     
    till, May 16, 2018
    #2
  3. Thomas Schachtner

    Thomas Schachtner New Member

    This part is working fine.
    I can access the file but cert renewal still doesn't work.
     
    Thomas Schachtner, May 17, 2018
    #3
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    When this works now, then renewal must work too.
     
    till, May 17, 2018
    #4
  5. Thomas Schachtner

    Thomas Schachtner New Member

    Unfortunately, it does not.
    I already tried that. The same error messages appear.
    Could it be that the http challenge cannot be created for some reason?
     
    Thomas Schachtner, May 17, 2018
    #5
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    Quite unlikely as this is run as root. But you should take a look into the letsencyrpt.log file and you might want to update certbot, just to be sure that there is no error in certbot. and you can try to access hello.txt from outside as well e.g. with a browser from your desktop.
     
    till, May 17, 2018
    #6
  7. Curtis Maurand

    Curtis Maurand Member

    Hello,
    I'm having this same issue. getting the same error. I created the folder manually. put a hello.txt file in there. The full path is:
    /var/www/clients/client7/web49/web/.well-known/acme-challenge/hello.txt (I copied the path from the terminal)
    getting 404 errors. As near as I can tell the .well-known/acme-challenge folder isn't even being created. after checking the for letsencrypt on this particular site, I watched to see if the folder was ever created and it didn't seem to be as I repeatedly did a "ls .w*" while in the web folder. This is a setup that was working before, but I had to rebuild from scratch and lay in configs. I ran the php -q update from the install folder yesterday and had to redo all of the letsencrypt configs for postfix and dovecot as the reconfigure services overwrote my changes. letsencrypt works if I spin up a standalone webserver which is how I got the mail server certs working and the ispconfig console working. I also tried making the folder world writable which didn't work, either.

    the system is otherwise working fine except for letsencrypt. Any help would be appreciated. Ispconfig version is 3.1.15p2

    Seems to have something to do with the ".well-known" as soon as I created the path without the . in front of well-known, I can get to the hello.txt file
     
    Last edited: Nov 14, 2019
    Curtis Maurand, Nov 14, 2019
    #7
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    There sould not be such a folder, there should never be a folder .well-known/acme-challenge/ inside a web folder of a site, the folder and path you created is wrong and needs to be removed. Please see my post #2 in this thread, it shows the exact command and path that you have to use for this test.

    That's not the case, you just did the test wrong as I described above and this must lead to wrong results.
     
    till, Nov 15, 2019
    #8
  9. Curtis Maurand

    Curtis Maurand Member

    I'm assuming that I replace the test1.example.com with the actual website name? i issued "echo testing >/usr/local/ispconfig/interface/acme/.well-known/acme-challenge/hello.txt" and I get a 404 not found error. Something must be slightly wrong.
     
    Curtis Maurand, Nov 15, 2019
    #9
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    yes

    first, check if you really created the hello.txt file. If you still can't reach it, then check the options tab of that site and if exists .htaccess files in the site, it might be that you are rewriting or blocking access to the file there.
     
    till, Nov 15, 2019
    #10
  11. Curtis Maurand

    Curtis Maurand Member

    The file is there.
    ls /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/hello.txt
    /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/hello.txt
    getting 404 errors.

    I did find a .htaccess file there for wordpress and moved it out to the folder above web (/var/www/clients/client7/web50/) but still makes no difference. I then moved it up one more level into client7 and it still made no difference. nothing in the options or custom php.ini. It is set to use php7-fpm
    I did some digging around in the apache conf folder and found a couple of rewrite rules in a couple of configs (specific websites and not ispconfig confs) that I commented out. I also found a letsencrypt-apache.sh that pointed to something in /var/lib and disabled it. I also found something called dehydrated.conf in the conf-available and changed it to point /usr/local/ispconfig/interface/acme/.well-known/acme-challenge and now I'm getting a forbidden error. At least it's different. Seems to me that there needs to be some sort of general alias to point to that folder, though.
     
    Curtis Maurand, Nov 15, 2019
    #11
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    At least a change in behaviours, that's good for the start. The config file that does the 'magic' in redirecting the requests is the ispconfig.conf file in the apache sites-enabled folder, there you find the alias for '/.well-known/acme-challenge' to '/usr/local/ispconfig/interface/acme/.well-known/acme-challenge' but there must be something on your system which interferes with that and prevets that the test file can be reached.
     
    till, Nov 15, 2019
    #12
  13. Curtis Maurand

    Curtis Maurand Member

    ah. got it. disabled dehydrated which pointed things to locations it shouldn't have. Things are working, now. Thanks for all your help. It's much appreciated.
     
    Curtis Maurand, Nov 15, 2019
    #13
    till likes this.
  14. RomeoD

    RomeoD New Member

    About the same problem here.
    Enabling SSL with Let's Encrypt SSL no longer works for new domains introduced.

    I mention that: domains already existing on the server automatically renew their SSL certificate.

    OS - Ubuntu 18.04.3 LTS

    ISPConfig version - 3.1.15p2

    certbot 0.27.0

    Python 2.7.15rc1

    hello.txt is working

    From letsencrypt log:

    Domain: ...... Type: connection Detail: Fetching ...../.well-known/acme-challenge/.....: Timeout during connect (likely firewall problem)

    To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you're using the webroot plugin, you should also verify that you are serving files from the webroot path you provided. 2019-11-25 04:13:16,409DEBUG:certbot.error_handler:Encountered exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations self._respond(aauthzrs, resp, best_effort) File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 155, in _respond self._poll_challenges(aauthzrs, chall_update, best_effort) File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 226, in _poll_challenges raise errors.FailedChallenges(all_failed_achalls) certbot.errors.FailedChallenges: Failed authorization procedure. .................(http-01): urn:ietfarams:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching ..............well-known/acme-challenge/...: Timeout during connect (likely firewall problem)

    2019-11-25 04:13:16,410EBUG:certbot.error_handler:Calling registered functions 2019-11-25 04:13:16,410:INFO:certbot.auth_handler:Cleaning up challenges 2019-11-25 04:13:16,410EBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/.... 2019-11-25 04:13:16,411DEBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/........ 2019-11-25 04:13:16,412DEBUG:certbot.plugins.webroot:All challenges cleaned up 2019-11-25 04:13:16,412DEBUG:certbot.log:Exiting abnormally: Traceback (most recent call last): File "/usr/bin/letsencrypt", line 11, in load_entry_point('certbot==0.27.0', 'console_scripts', 'certbot')() File "/usr/lib/python3/dist-packages/certbot/main.py", line 1364, in main return config.func(config, plugins) File "/usr/lib/python3/dist-packages/certbot/main.py", line 1254, in certonly lineage = _get_and_save_cert(le_client, config, domains, certname, lineage) File "/usr/lib/python3/dist-packages/certbot/main.py", line 120, in _get_and_save_cert lineage = le_client.obtain_and_enroll_certificate(domains, certname) File "/usr/lib/python3/dist-packages/certbot/client.py", line 391, in obtain_and_enroll_certificate cert, chain, key, _ = self.obtain_certificate(domains) File "/usr/lib/python3/dist-packages/certbot/client.py", line 334, in obtain_certificate orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names) File "/usr/lib/python3/dist-packages/certbot/client.py", line 370, in _get_order_and_authorizations authzr = self.auth_handler.handle_authorizations(orderr, best_effort) File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 82, in handle_authorizations self._respond(aauthzrs, resp, best_effort) File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 155, in _respond self._poll_challenges(aauthzrs, chall_update, best_effort) File "/usr/lib/python3/dist-packages/certbot/auth_handler.py", line 226, in _poll_challenges raise errors.FailedChallenges(all_failed_achalls) certbot.errors.FailedChallenges: Failed authorization procedure. .............(http-01): urn:ietfarams:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching ............/.well-known/acme-challenge/..........: Timeout during connect (likely firewall problem)
     
    Last edited: Nov 25, 2019
    RomeoD, Nov 25, 2019
    #14
  15. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    Looks like the new domains do not work. Can you access your web server with the new domains from outside your local network?
     
    Taleman, Nov 25, 2019
    #15
  16. RomeoD

    RomeoD New Member

    Yes, this work ok (externally tested - with mobile wireless off and 4G on).
    .... /.well-known/acme-challenge/hello.txt
     

    Attached Files:

    Last edited: Nov 25, 2019
    RomeoD, Nov 25, 2019
    #16
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    Do you have any web firewalls in front of the system or a service like cloudflare which might block the requests?
     
    till, Nov 25, 2019
    #17
  18. RomeoD

    RomeoD New Member

    Access is not blocked, I tested from the outside, from 4G mobile, I can access text.txt (you can also try, you have the domain in the attachment above).
     
    RomeoD, Nov 25, 2019
    #18
  19. till

    till Super Moderator Staff Member ISPConfig Developer

    Strange that LE complains then. Is certbot up to date?
     
    till, Nov 25, 2019
    #19
  20. RomeoD

    RomeoD New Member

    certbot 0.27.0
    Python 2.7.15rc1

    But in logs i see python3 directory ! Hm...
    load_entry_point('certbot==0.27.0', 'console_scripts', 'certbot')()
    File "/usr/lib/python3

     
    Last edited: Nov 25, 2019
    RomeoD, Nov 25, 2019
    #20
Page 1 of 2

Share This Page