Latest Ispconfig running NGINX on Debian 10.13 One of my domains is having a problem getting Let's Encrypt to work. errors: /etc/letsencrypt/live/MyDomain.com/fullchain.pem (failure) . . 2023-05-13 03:04:15,005:ERROR:certbot._internal.log:5 renew failure(s), 0 parse failure(s) I have attempted to remove and reload this cert several times without luck. What is the best way to resolve this problem? How do I tell if I have introduced a rate limiting problem?
Please follow this procedure to find the reason for the LE issue: https://forum.howtoforge.com/threads/lets-encrypt-error-faq.74179/
Thanks for the reply. I am having a hard time with the 9 steps and/or they are in many cases not applicable to my current configuration. All sites/domains are working except for 1 so It seems to me that these steps are for reconfiguring the whole server and not just troubleshooting 1 site/domain that is not working. It also appears that I am generating some type of cert. I do not know if it is from LE or not. Additionally when I get to: Create new ISPConfig SSL certificate (yes,no) [no]: I assume that this is not a LE cert so the answer is still no. This is a single site issue. I am not understanding how the 9 steps for an apache server relate so a host sith a single site LE failure. Yes I appear to be using certbot.
The steps are applicable to any ISPConfig system. if you say they are not applicable to your system, then this must be a server without ISPConfig. In this case, please post in the general support forum here as priority support is for ISPConfig systems only. This assumption is wrong too. These steps are for any kind of LE SSL problem, no matter if it affects a single site or all sites. Follow the FAQ until the end. This is about the ISPConfig installer, it is not related to the FAQ and the FAQ did not tell you tu run this. Follow the let#s encrypt FAQ if you like to know why your website cert fails. Great, so you know the right way how to solve it already, here the link again: https://forum.howtoforge.com/threads/lets-encrypt-error-faq.74179/ Follow the steps, one after another. And as the FAQ clearly tells you, if you were not able to locate the problem yourself with the first steps, you must post the debug output if you like to get help. You are just wasting your time and my time by not following the FAQ, so if you want to get your issue resolved, start to follow it to the letter. I have no idea why some people think that they must question and discuss the steps instead of just following them. I have written most of the code in ISPConfig and I'm doing this for more than 20 years, so I know what I'm doing here and why these steps in the FAQ exists.
1) Yes, I am using certbot 2) Check that the Let's encrypt client 'certbot' is updated. Not sure how this is done? certbot --version and certbot -v did not work. Not knowing the proper way to find this information out I simply installed certbot on the server >apt install certbot -y >certbot --version certbot --version >certbot 0.31.0 3) I just updated IPConfig I can only assume that it 3.2.9p1. If I go to tools I can not find anything that references software version. Where did this information go? 4) system > server convig > ssl > Skip Lets Encrypt Check ---- Is not checked, 6) cloudflare is not being used on this domain. 6) DNS records look good 7) NGINX not Apache 8) I kept all of the defaults when updating ISPconfig to 3.2.9p1 9) Server Migration Mode is not checked. Still have the problem on a single site/domain. Tried 2 browsers all other sites have LE SSL lock..
The software version is under Help and not Tools. Really bad idea, the FAQ did not tell you to do that. You likely destroyed your LE setup now, so instead of having a small problem with a single domain, you now have likely a huge problem with all domains as your system likely uses acme.sh when certbot is not installed and installing acme.sh and certbot together messes things up badly. We can just try now to limit the damage and use debug mode to see what the original problem is and then try to fix the new problems you caused by not following the FAQ. And that's what the FAQ tells you to do in this case: So you still did not follow the FAQ until the end.
I have confirmed that I have ISPConfig Version: 3.2.9p1 acme.sh is not on this system. All previously working domains still appear to be OK. /usr/local/ispconfig/server/server.sh finished server.php. I do not know why one of my domains does not wotk with LE.
Post the output of: ls -la /root/.acme.sh/ So you still missed enabling debug mode, read the instructions on how to enable debug mode carefully and do not leave 50% out. Plus you did not do what the FAQ tells you to do, which is to enable let's encrypt checkbox again and save the change before running server.sh. So here again the text from FAQ that you should follow: Please read the whole text and follow all instructions found there.
root@mr2:~# ls -la /root/.acme.sh/ ls: cannot access '/root/.acme.sh/': No such file or directory Monitor > System State (All Servers) > Show System-Log) No results root@mr2:~# /usr/local/ispconfig/server/server.sh 15.05.2023-16:19 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. 15.05.2023-16:19 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock finished server.php.
You enabled debug mode now, but you still missed to enable let's encrypt checkbox in the website again before running server.sh.
The debug output shows that you did not enable it in this debug run. So the Let's encrypt checkbox is enabled, and it stays enabled? If that's the case, then the site must have a valid LE cert, unless files were manually deleted after they were created by ISPConfig. What you can do is untick the checkbox, press save, tick it again, run server.sh and post the debug output.
root@mr2:~# /usr/local/ispconfig/server/server.sh 16.05.2023-18:01 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. 16.05.2023-18:01 - DEBUG [server:177] - Found 2 changes, starting update process. 16.05.2023-18:01 - DEBUG [plugins.inc:118] - Calling function 'ssl' from plugin 'nginx_plugin' raised by event 'web_domain_update'. 16.05.2023-18:01 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'nginx_plugin' raised by event 'web_domain_update'. 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr -i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr +i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: df -T '/var/www/clients/client1/web3'|awk 'END{print $2,$NF}' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: which 'setquota' 2> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: setquota -u 'web3' '0' '0' 0 0 -a &> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: setquota -T -u 'web3' 604800 604800 -a &> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr +i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [nginx plugin.inc:1394] - Enable SSL for: MyDomain.com 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: nginx -V 2>&1 | grep 'built with OpenSSL' | sed 's/.*built\([a-zA-Z ]*\)OpenSSL \([0-9.]*\).*/\2/' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: nginx -V 2>&1 | grep 'running with OpenSSL' | sed 's/.*running\([a-zA-Z ]*\)OpenSSL \([0-9.]*\).*/\2/' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: which 'nginx' 2> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [nginx plugin.inc:1623] - Enable TLS 1.3 for: MyDomain.com 16.05.2023-18:01 - DEBUG [nginx plugin.inc:1916] - Writing the vhost file: /etc/nginx/sites-available/MyDomain.com.vhost 16.05.2023-18:01 - DEBUG [nginx plugin.inc:3042] - Writing the PHP-FPM config file: /etc/php/7.3/fpm/pool.d/web3.conf 16.05.2023-18:01 - DEBUG [services.inc:56] - Calling function 'restartPHP_FPM' from module 'web_module'. 16.05.2023-18:01 - DEBUG [system.inc:2082] - Trying to use Systemd to restart service 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: systemctl is-enabled 'php7.3-fpm' 2>&1 - return code: 0 16.05.2023-18:01 - DEBUG [web module.inc:316] - Restarting php-fpm: systemctl reload php7.3-fpm.service 16.05.2023-18:01 - DEBUG [nginx plugin.inc:2017] - nginx status is: running 16.05.2023-18:01 - DEBUG [services.inc:56] - Calling function 'restartHttpd' from module 'web_module'. 16.05.2023-18:01 - DEBUG [system.inc:2082] - Trying to use Systemd to restart service 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: systemctl is-enabled 'nginx' 2>&1 - return code: 0 16.05.2023-18:01 - DEBUG [web module.inc:236] - Checking nginx configuration... 16.05.2023-18:01 - DEBUG [web module.inc:239] - nginx configuration ok! 16.05.2023-18:01 - DEBUG [web module.inc:246] - Restarting httpd: systemctl restart nginx.service 16.05.2023-18:01 - DEBUG [nginx plugin.inc:2020] - nginx restart return value is: 0 16.05.2023-18:01 - DEBUG [nginx plugin.inc:2027] - nginx online status after restart is: running 16.05.2023-18:01 - DEBUG [modules.inc:240] - Processed datalog_id 4079 16.05.2023-18:01 - DEBUG [plugins.inc:118] - Calling function 'ssl' from plugin 'nginx_plugin' raised by event 'web_domain_update'. 16.05.2023-18:01 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'nginx_plugin' raised by event 'web_domain_update'. 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr -i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr +i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: df -T '/var/www/clients/client1/web3'|awk 'END{print $2,$NF}' - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: which 'setquota' 2> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: setquota -u 'web3' '0' '0' 0 0 -a &> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: setquota -T -u 'web3' 604800 604800 -a &> /dev/null - return code: 0 16.05.2023-18:01 - DEBUG [system.inc:2399] - safe_exec cmd: chattr +i '/var/www/clients/client1/web3' - return code: 0 16.05.2023-18:01 - DEBUG [letsencrypt.inc:156] - LE version is 0.31.0, so using certificates command and --cert-name instead of --expand 16.05.2023-18:01 - DEBUG [letsencrypt.inc:431] - Create Let's Encrypt SSL Cert for: MyDomain.com 16.05.2023-18:01 - DEBUG [letsencrypt.inc:432] - Let's Encrypt SSL Cert domains: 16.05.2023-18:01 - DEBUG [system.inc:1819] - exec: /usr/bin/certbot certonly -n --text --agree-tos --cert-name MyDomain.com --authenticator webroot --server https://acme-v02.api.letsencrypt.org/directory --rsa-key-size 4096 --email [email protected] --webroot-map '{"MyDomain.com":"\/usr\/local\/ispconfig\/interface\/acme","www.MyDomain.com":"\/usr\/local\/ispconfig\/interface\/acme"}' Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator webroot, Installer None Attempting to parse the version 1.9.0 renewal configuration file found at /etc/letsencrypt/renewal/MyDomain.com.conf with version 0.31.0 of Certbot. This might not work. Attempting to parse the version 1.9.0 renewal configuration file found at /etc/letsencrypt/renewal/MyDomain.com.conf with version 0.31.0 of Certbot. This might not work. Cert is due for renewal, auto-renewing... Renewing an existing certificate Performing the following challenges: http-01 challenge for www.MyDomain.com Waiting for verification... Cleaning up challenges Failed authorization procedure. www.MyDomain.com (http-01): urn:ietfarams:acme:error:connection :: The server could not connect to the client to verify the domain :: 69.195.135.34: Fetching http://www.MyDomain.com/.well-known/acme-challenge/SYN0ZgvRC4u5faR_C7cFdgkSwugNeQhvrpzIcunVEHo: Timeout during connect (likely firewall problem) 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Found the following matching certs: 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Certificate Name: MyDomain.com 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Domains: MyDomain.com www.MyDomain.com 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Expiry Date: 2022-10-19 07:04:38+00:00 (INVALID: EXPIRED) 16.05.2023-18:02 - DEBUG [letsencrypt.inc:476] - Found LE path is expired or invalid: 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Certificate Path: /etc/letsencrypt/live/MyDomain.com/fullchain.pem 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: Private Key Path: /etc/letsencrypt/live/MyDomain.com/privkey.pem 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 16.05.2023-18:02 - DEBUG [letsencrypt.inc:468] - LE CERT OUTPUT: 16.05.2023-18:02 - DEBUG [letsencrypt.inc:264] - Let's Encrypt Cert config path is: /etc/letsencrypt/renewal/MyDomain.com.conf. 16.05.2023-18:02 - WARNING - Let's Encrypt SSL Cert for: MyDomain.com could not be issued. 16.05.2023-18:02 - WARNING - /usr/bin/certbot certificates --domains MyDomain.com --domains www.MyDomain.com 16.05.2023-18:02 - DEBUG [letsencrypt.inc:528] - Let's Encrypt Cert file: /etc/letsencrypt/live/MyDomain.com/fullchain.pem exists. 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: ln -s '/etc/letsencrypt/live/MyDomain.com/privkey.pem' '/var/www/clients/client1/web3/ssl/MyDomain.com-le.key' - return code: 0 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: ln -s '/etc/letsencrypt/live/MyDomain.com/fullchain.pem' '/var/www/clients/client1/web3/ssl/MyDomain.com-le.crt' - return code: 0 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: ln -s '/etc/letsencrypt/live/MyDomain.com/chain.pem' '/var/www/clients/client1/web3/ssl/MyDomain.com-le.bundle' - return code: 0 16.05.2023-18:02 - DEBUG [nginx plugin.inc:1394] - Enable SSL for: MyDomain.com 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: nginx -V 2>&1 | grep 'built with OpenSSL' | sed 's/.*built\([a-zA-Z ]*\)OpenSSL \([0-9.]*\).*/\2/' - return code: 0 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: nginx -V 2>&1 | grep 'running with OpenSSL' | sed 's/.*running\([a-zA-Z ]*\)OpenSSL \([0-9.]*\).*/\2/' - return code: 0 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: which 'nginx' 2> /dev/null - return code: 0 16.05.2023-18:02 - DEBUG [nginx plugin.inc:1623] - Enable TLS 1.3 for: MyDomain.com 16.05.2023-18:02 - DEBUG [nginx plugin.inc:1916] - Writing the vhost file: /etc/nginx/sites-available/MyDomain.com.vhost 16.05.2023-18:02 - DEBUG [nginx plugin.inc:3042] - Writing the PHP-FPM config file: /etc/php/7.3/fpm/pool.d/web3.conf 16.05.2023-18:02 - DEBUG [services.inc:56] - Calling function 'restartPHP_FPM' from module 'web_module'. 16.05.2023-18:02 - DEBUG [system.inc:2082] - Trying to use Systemd to restart service 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: systemctl is-enabled 'php7.3-fpm' 2>&1 - return code: 0 16.05.2023-18:02 - DEBUG [web module.inc:316] - Restarting php-fpm: systemctl reload php7.3-fpm.service 16.05.2023-18:02 - DEBUG [nginx plugin.inc:2017] - nginx status is: running 16.05.2023-18:02 - DEBUG [services.inc:56] - Calling function 'restartHttpd' from module 'web_module'. 16.05.2023-18:02 - DEBUG [system.inc:2082] - Trying to use Systemd to restart service 16.05.2023-18:02 - DEBUG [system.inc:2399] - safe_exec cmd: systemctl is-enabled 'nginx' 2>&1 - return code: 0 16.05.2023-18:02 - DEBUG [web module.inc:236] - Checking nginx configuration... 16.05.2023-18:02 - DEBUG [web module.inc:239] - nginx configuration ok! 16.05.2023-18:02 - DEBUG [web module.inc:246] - Restarting httpd: systemctl restart nginx.service 16.05.2023-18:02 - DEBUG [nginx plugin.inc:2020] - nginx restart return value is: 0 16.05.2023-18:02 - DEBUG [nginx plugin.inc:2027] - nginx online status after restart is: running 16.05.2023-18:02 - DEBUG [modules.inc:240] - Processed datalog_id 4080 16.05.2023-18:02 - DEBUG [server:217] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
Ok, so finally here we have the output including the exact error: This means Let's encrypt tried to reach the domain but could not reach it. Likely, problems are that you block access in a firewall or you use Geo-blocking that prevents access from the location that Let#s encrypt hosts its servers.
None of the other domains have this problem. They all use the same firewall and I have never set up any form of GEO blocking.
It does not matter if other domains have this issue or not. LE reports to you clearly that access to this one domain is blocked in a way that it times out when LE tries to connect to it, and you will not get a new cert until you unblock access to it for LE servers. There are no issues in ISPConfig or the LE client here, LE client is executed properly but access from LE servers to this domain is blocked so it times out.
Is there a good test? A browser can access the domain. dig results look similar to working domain All DNS records look similar too. Is there a way to simply refresh a site?
No, as you do not have access to the systems from let#s encrypt to test from their side. This is good, but not comparable as you are in a different location. Let's encrypt will try to use IPv6 if available, so it might e.g. by that you block IPv6 while the domain has an IPv6 record and your browser uses IPv4, but that's not used by LE. Use a system like intodns.com to test it. There are many kinds of possible errors in DNS that you do not spot easily, like split-brain situations between DNS mirrors etc. You did that already according to the log. Any change in the settings of a site plus saving the settings refreshes it.
intodns.com results look similar across all domains. Is there anything that can be done fr the domain that does not appear to communicate with LE?
I mentioned a few things above like testing IPv6 and trying to test from a different location e.g. by using a VPN and taking care that you test with HTTP and not HTTPS, as the validation token is accessed by Let's encrypt via HTTP on port 80. or you contact Thom from ISPConfig business support to check that for you https://www.ispconfig.org/get-support/?type=ispconfig
