Currently all my SSL certificates are out of date. The reason for this seems to be this issue: https://git.ispconfig.org/ispconfig/ispconfig3/issues/4380 There is also a priority thread about this. I want to solve this ASAP, but don't understand the steps to take. Could someone explain the steps to solve this manually? THese are the errors I get from the letsencrypt log: 2017-01-11 02:00:23,344EBUG:certbot.plugins.webroot:Creating root challenges validation dir at /usr/local/ispconfig/interface/acme/.well-known/acme-challenge 2017-01-11 02:00:23,388EBUG:certbot.plugins.webroot:Attempting to save validation to /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/dFv... FailedChallenges: Failed authorization procedure. sub.domain.nl (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://sub.domain.nl/.well-known/acme-challenge/dFv... <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <title>Page not" I assume the .well-known path is somewhere rewritten to the /user/local path instead of to the root of the /web/ folder for that domain??
The issue you linked is not about failing renewal, it is about successfully renewal but a wrong symlink and it is limited for a special configuration using relative symlinks, so it does not apply to a default ISPConfig setup. According to your log, the renewal fails because lets encrypt is unable to reach a sub domain, so your problem is different Your log file says that lets encrypt tried to reach sub.domain.nl from outside to verify the SSL cert and this domain could not be reached. So the first step is to check if DNS for sub.domain.nl is correct and pointing to the right server
Ah. Sorry for the mix-up then The subdomain does exist: I can reach it from the outside and about three months ago I was able to request the LetsEncrypt certificates. Furthermore: - subdomain is a proxy for another webserver - http is directed to https This is in the vhost for port 80: RewriteEngine on RewriteCond %{REQUEST_URI} ^/\.well-known/acme-challenge/ RewriteRule ^ - [END] RewriteCond %{HTTPS} off RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L] What I don't understand is that the request is to http://sub.domain.nl/.well-known/acme-challenge/dFv..., but that the log says that the actual location is at /usr/isponfig... Is that somewhere rewritten / linked, as there is no .well-known folder in the rootfolder of the domain. I disabled the proxy in the *.80 vhost config as I thought that that could be the problem, but that didn't solve the renewal problem.
Yes, the path gets rewritten to a central directory where all tokens for all domains get created, you can find the rewrite rule in the apache ispconfig.conf file.
Ok. I tested access to http://sub.domain.nl/.well-known/acme-challenge/ I get a 403 and an ISPConfig generated screen with: The following error occurred: You are not permitted to access the requested URL. Please contact the webmaster with any queries. This generates the following error in the subdomains access.log file: [Wed Jan 11 14:22:42.699626 2017] [autoindex:error] [pid 15385] [client 172.27.10.80:63789] AH01276: Cannot serve directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/: No matching DirectoryIndex (index.html,inde$ The access from last night was (access.log file for sub domain) a 404 result: 66.133.109.36 - - [11/Jan/2017:03:00:27 +0100] "GET /.well-known/acme-challenge/pgAzyYZTi9mllkG3RVjWztUCAzjBHkxTdq10X9F58jU HTTP/1.1" 404 927 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +https://www.letsencrypt.org)" So the main folder gives me an permission error, but LetsEncrypt validation gets a 404. What's the next thing I could check??
The permission error is fine as you should not be able to get a file list for that folder (and that's what you request by accessing it with a browser). letsencrypt creates a token file and then tries to access it in the folder, so if you want to try what letsencrypt is doing, then you must create a file in that folder and then try to access that file with a browser. My guess is that it is related to the proxy redirect, I know that you disabled it but there must be still some config that causes the request from letsencrypt to be redirected to a wrong folder or server as it worked at the time you created the letsencrypt cert, so the change must be made after that date. did you try a manual letsencrypt renewal after you disabled the proxy? The command is either: certbot -n renew or letsencrypt -n renew
I don't want to solve it manually, as this does not solve the problem. I did make a file named 'testfile' with the contents "hello" in /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/test# and can access that file via: http://sub.domain.nl/.well-known/acme-challenge/test/testfile, where the "hello" is shown in the browser window! THis works for ALL the disables proxies. It doens't for enabled proxies, so disabling the proxies in the *.80 vhost configuration is essential for this to work! I can also access the empty.dir file (it is downloaded by my browser). So no problems with the proxy. But what's the problem then??
That's the same command that ispconfig runs once a night. But you can wait until tomorrow of course until the command has been run by the nightly cronjob.
Aha, then I might try it manually!. Next thing I see in the logging: - dns-01 challenge fails - challenge files are removed - then the http-01 challenge fails with a 404 on that file that is just removed by the failing dns-01 challenge 2017-01-11 02:00:26,810:INFO:certbot.auth_handler:Cleaning up challenges 2017-01-11 02:00:26,810EBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/dFvpmcL2MnrXEfnVp57WQ9e7xmFEOyKdFUO-Bjw1GHU 2017-01-11 02:00:26,811:INFO:certbot.plugins.webroot:Unable to clean up challenge directory /usr/local/ispconfig/interface/acme/.well-known/acme-challenge 2017-01-11 02:00:26,811EBUG:certbot.plugins.webroot:Error was: [Errno 39] Directory not empty: '/usr/local/ispconfig/interface/acme/.well-known/acme-challenge' 2017-01-11 02:00:26,811:WARNING:certbot.renewal:Attempting to renew cert from /etc/letsencrypt/renewal/sub.domain.nl-0001.conf produced an unexpected error: Failed authorization procedure. sub.domain.nl (http-01): urn:acme: error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://sub.domain.nl/.well-known/acme-challenge/dFvpmcL2MnrXEfnVp57WQ9e7xmFEOyKdFUO-Bjw1GHU: " <!DOCTYPE html> <html lang="en"> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> <title>Page not". Skipping. Is that something?
DNS challenge is not used, so it is ok that it fails, but the http challenge should work, that's the token download attempt. Does it still fail for the domain where you removed the proxy?
Yes. I just added the logging in my previous post. Look at this line: 2017-01-11 02:00:26,810EBUG:certbot.plugins.webroot:Removing /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/dFvpmcL2MnrXEfnVp57WQ9e7xmFEOyKdFUO-Bjw1GHU That is done before the http-01 challenge so it seems! It is a subdomain where I removed the proxy, ie where I proved that access to the testfile is working.
Disables https redirect. At the same time ssl and letsencrypt where unchecked. Can't check letsencrypt anymore. If I enable it, it is disabled again after saving... THe only thing I did after the initial letsencrypt is upgrade ISPCOnfig from 3.1 to the 3.1.p.1 And I can't find letsencrypt and certbot commands to manually renew. Where are these located?
Status update of this problem The proxy is definately the source of the problem. Completely removing the proxy (in the website options tab in ISPConfig) resulted in a renewal of the LetsEncrypt certificate. After that, I tried to find a solution that would result in: - No proxy for port 80 - A proxy for port 443 My first try was (with the incredible bad Apache documentation) to use a check on the port number: <If "%{SERVER_PORT} == '443'"> </If> But it appears that the ProxyPass directive is NOT accepted within that If block So I changed that using the <Location> block. Below should disable the proxy for LetsEncrypt challenges: <Location "/.well-known/acme-challenge/"> ProxyPass "!" ProxyPassReverse "!"</Location> I still have to run some tests to make sure that this works. The problem is that if I add a domain, and tick the SSL and LetsEncrypt checkboxes and continue to the other tab to enter the proxy details, ISPConfig already starts to issue the certificate (the red circle at the top is already blinking while I'm still entering data for the domain and I haven't hit the "Save" button yet!!!). So I'm not 100% sure that the above solution is indeed the right one! No Idea if I should use a wildcard orso...
Last night's log in the sub.domain.nl access log, 404's as before with a proxied subdomain: 66.133.109.36 - - [25/Jan/2017:03:02:20 +0100] "GET /.well-known/acme-challenge/rC6... HTTP/1.1" 404 1547 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; $ 66.133.109.36 - - [25/Jan/2017:03:02:24 +0100] "GET /.well-known/acme-challenge/Acen... HTTP/1.1" 404 1548 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; $ I Changed the proxy settings yesterday afternoon and re-enabled HTTPS in ISPConfig giving this log: 66.133.109.36 - - [25/Jan/2017:16:12:10 +0100] "GET /.well-known/acme-challenge/TEpe... HTTP/1.1" 200 450 "-" "Mozilla/5.0 (compatible; Let's Encrypt validation server; +$ But still, the certificate is old... NO renewal has taken place! The latest access logfile doesn't contain any GET from LetsEncrypt. However: This subdomain has a -0001.conf file in the /etc/letsencrypt/renewal folder. The other subdomains don't have that. The LetsEncrypt logfile tells me that renewal has taken place: 2017-01-26 02:00:30,859EBUG:certbot.storage:Writing new private key to /etc/letsencrypt/archive/sub.domain.nl-0001/privkey2.pem. 2017-01-26 02:00:30,859EBUG:certbot.storage:Writing certificate to /etc/letsencrypt/archive/sub.domain.nl-0001/cert2.pem. 2017-01-26 02:00:30,859EBUG:certbot.storage:Writing chain to /etc/letsencrypt/archive/sub.domain.nl-0001/chain2.pem. 2017-01-26 02:00:30,859EBUG:certbot.storage:Writing full chain to /etc/letsencrypt/archive/sub.domain.nl-0001/fullchain2.pem. 2017-01-26 02:00:31,669EBUG:certbot.storage:Writing new config /etc/letsencrypt/renewal/sub.domain.nl-0001.conf.new. So, what's going wrong in this case???
New day, new results! Another domain went well last night: the certificate was renewed However, https did'nt work for that domain It appears that ISPConfig simply 'forgets' to write the *:443 part in the vhost file after some change in the configuration of the domain. Only the *:80 part is present. I have to disable and enable SSL and LetsEncrypt checks before ISPConfig does write a complete vhost file again. So, the renewal was with a domain that had only port 80 enabled. The proxy & location statements where in it though, so I hope that this solution does work. Not writing the port 443 part must be a bug I guess... I'm running the latest ISPConfig v3.1.2
