Running ISPConfig 3.2 on Debian 10 smoothly for months, my Apache2 webserver decided to stop working and fails to start. Obviously it is essential to have Apache2 running. The logs show only Certificate and private key xxx:8080:0 from /usr/local/ispconfig/interface/ssl/ispserver.crt and /usr/local/ispconfig/interface/ssl/ispserver.key do not match (edit) Checking the acme log, it appears that the SSL for domain I use to run ispconfig failed to renew: vibel.be:Verify error:Invalid response from http://vibel.be/.well-known/acme-challenge (which is not on the system) Any help on getting my apacheserver back online would be greatfully accepted.
So the server uses a hostname that points to another system? Does the command: hostname or hostname -f returns this 'wrong' hostname?
What do you mean with: Try the following: Open /etc/apache2/sites-available/ispconfig.vhost Comment this out: Code: # SSL Configuration SSLEngine On SSLProtocol All -SSLv3 -TLSv1 -TLSv1.1 SSLCertificateFile /usr/local/ispconfig/interface/ssl/ispserver.crt SSLCertificateKeyFile /usr/local/ispconfig/interface/ssl/ispserver.key #SSLCACertificateFile /usr/local/ispconfig/interface/ssl/ispserver.bundle SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECD$ SSLHonorCipherOrder On Start apache: Code: systemctl start apache2 Run a force update to retrieve a new cert. Share the output of the update script with us: Code: ispconfig_update.sh --force
Thank you. After ALSO disabling these in apps.vhost, I got Apache2 running again. However, I cannot connect to ISPConfig (on port 8080). Certificate and private key nlhb52619ds.vibel.be:8081:0 from /usr/local/ispconfig/interface/ssl/ispserver.crt and /usr/local/ispconfig/interface/ssl/ispserver.key do not match (edit) As a result of this new error I went looking and found the port 8081 in apps.vhost and I commented out the SSL configuration there too. And re: "what do you mean with" http://vibel.be/.well-known/acme-challenge <-- this path does not exist, it is not present. The acme log gave this as reason for the failed response of vibel.be (proceeding with iispconfig_update.sh --force)
You did not share the output of ispconfig_update.sh --force. It should show an error message if certificate creation fails. https://www.howtoforge.com/community/threads/lets-encrypt-error-faq.74179/ https://www.howtoforge.com/community/threads/please-read-before-posting.58408/
That#s a virtual path, not real path in filesystem. So it shall not exist in the website. This virtual path is mapped to /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/
Using apache for certificate validation acme.sh is installed, overriding certificate path to use /root/.acme.sh/nlhb52619ds.vibel.be [Mon 19 Jul 2021 02:56:25 PM CEST] nlhb52619ds.vibel.be:Verify error:Invalid response from http://nlhb52619ds.vibel.be/.well-known/acme-challenge/GV8aJMsr_yBZYGPFTrxlPlP6mPJLO54B_T-5lbktIbM [2606:4700:3036::6815:1120]: [Mon 19 Jul 2021 02:56:25 PM CEST] Please check log file for more details: /var/log/ispconfig/acme.log Issuing certificate via acme.sh failed. Please check that your hostname can be verified by letsencrypt Could not issue letsencrypt certificate, falling back to self-signed. Generating a RSA private key ...........................................................................................................++++ .++++ writing new private key to '/usr/local/ispconfig/interface/ssl/ispserver.key' Letsencrypt searches for /.well-known/acme-challenge, which path does not exist ... AND issueingvia acme.sh failed.... This makes accessing ispconfig impossible...
That's not what the message above says. It says that a token which it created in the folder /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/ is not reachable from outside of your server by http://. You can test it like this. Run the command: touch /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/test.txt you must now be able to access this file with this URL to receive a valid LE SSL cert: http://nlhb52619ds.vibel.be/.well-known/acme-challenge/test.txt if this fails, then check if DNS really points to the right server (IPv4 and IPv6) and that you don't block port 80 in a firewall.
You were right assuming that "if this fails, then check if DNS really points to the right server". Cloudflare DNS proxied the IP address. I had to correct that to enable it to directly connect to the IP address and checked that it does. However running [ispconfig_update.sh --force] again after verifying that the test file is reacheable and readable, failed again [Mon 19 Jul 2021 03:46:51 PM CEST] nlhb52619ds.vibel.be:Verify error:Fetching http://nlhb52619ds.vibel.be/.well-known/acme-challenge/H97oxzhqtgX3i5hFEFcVgnJETHZZzYOtAnEoag78PnI: Connection refused [Mon 19 Jul 2021 03:46:51 PM CEST] Please check log file for more details: /var/log/ispconfig/acme.log Issuing certificate via acme.sh failed. Please check that your hostname can be verified by letsencrypt Could not issue letsencrypt certificate, falling back to self-signed. Generating a RSA private key
Did you comment out the SSL config lines again like @Th0m suggested before running the forced upgrade again? If no, please try the full procedure again by commenting out these lines, then doing a forced update.
I redid the commenting out of the SSL config lines on ispconfig.vhost AND the apps.vhost, then restarted apache2 and then sudo ispconfig_update.sh --force --> same result Checking / creating certificate for nlhb52619ds.vibel.be Using certificate path /root/.acme.sh/nlhb52619ds.vibel.be Using apache for certificate validation acme.sh is installed, overriding certificate path to use /root/.acme.sh/nlhb52619ds.vibel.be [Mon 19 Jul 2021 04:21:18 PM CEST] nlhb52619ds.vibel.be:Verify error:Fetching http://nlhb52619ds.vibel.be/.well-known/acme-challenge/YqMGoOgMSeSTc_UZzkm9qK6mJSuWnfAlelT2QNe1ukY: Connection refused But at least, apache2 is up again and the websites are live. Only access to the ISPConfig interface is prohibited (or accessible through forced https access exception) Here is the ispconfig acme log [Mon 19 Jul 2021 05:51:37 PM CEST] Running cmd: issue [Mon 19 Jul 2021 05:51:37 PM CEST] _main_domain='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:37 PM CEST] _alt_domains='no' [Mon 19 Jul 2021 05:51:37 PM CEST] Using config home:/root/.acme.sh [Mon 19 Jul 2021 05:51:37 PM CEST] default_acme_server='https://acme-v02.api.letsencrypt.org/directory' [Mon 19 Jul 2021 05:51:37 PM CEST] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory' [Mon 19 Jul 2021 05:51:37 PM CEST] DOMAIN_PATH='/root/.acme.sh/nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:37 PM CEST] Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory [Mon 19 Jul 2021 05:51:37 PM CEST] _init api for server: https://acme-v02.api.letsencrypt.org/directory [Mon 19 Jul 2021 05:51:37 PM CEST] GET [Mon 19 Jul 2021 05:51:37 PM CEST] url='https://acme-v02.api.letsencrypt.org/directory' [Mon 19 Jul 2021 05:51:37 PM CEST] timeout= [Mon 19 Jul 2021 05:51:37 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:37 PM CEST] ret='0' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_NEW_AUTHZ [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf' [Mon 19 Jul 2021 05:51:38 PM CEST] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce' [Mon 19 Jul 2021 05:51:38 PM CEST] Le_NextRenewTime [Mon 19 Jul 2021 05:51:38 PM CEST] Using CA: https://acme-v02.api.letsencrypt.org/directory [Mon 19 Jul 2021 05:51:38 PM CEST] _on_before_issue [Mon 19 Jul 2021 05:51:38 PM CEST] _chk_main_domain='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:38 PM CEST] _chk_alt_domains [Mon 19 Jul 2021 05:51:38 PM CEST] Le_LocalAddress [Mon 19 Jul 2021 05:51:38 PM CEST] d='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:38 PM CEST] Check for domain='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:38 PM CEST] _currentRoot='/usr/local/ispconfig/interface/acme' [Mon 19 Jul 2021 05:51:38 PM CEST] d [Mon 19 Jul 2021 05:51:38 PM CEST] _saved_account_key_hash is not changed, skip register account. [Mon 19 Jul 2021 05:51:38 PM CEST] Read key length: [Mon 19 Jul 2021 05:51:38 PM CEST] _createcsr [Mon 19 Jul 2021 05:51:38 PM CEST] Single domain='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:38 PM CEST] Getting domain auth token for each domain [Mon 19 Jul 2021 05:51:38 PM CEST] d [Mon 19 Jul 2021 05:51:38 PM CEST] url='https://acme-v02.api.letsencrypt.org/acme/new-order' [Mon 19 Jul 2021 05:51:38 PM CEST] payload='{"identifiers": [{"type":"dns","value":"nlhb52619ds.vibel.be"}]}' [Mon 19 Jul 2021 05:51:38 PM CEST] RSA key [Mon 19 Jul 2021 05:51:38 PM CEST] HEAD [Mon 19 Jul 2021 05:51:38 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce' [Mon 19 Jul 2021 05:51:38 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g -I ' [Mon 19 Jul 2021 05:51:39 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:39 PM CEST] POST [Mon 19 Jul 2021 05:51:39 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-order' [Mon 19 Jul 2021 05:51:39 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:40 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:40 PM CEST] code='201' [Mon 19 Jul 2021 05:51:40 PM CEST] Le_LinkOrder='https://acme-v02.api.letsencrypt.org/acme/order/124272533/11192376274' [Mon 19 Jul 2021 05:51:40 PM CEST] Le_OrderFinalize='https://acme-v02.api.letsencrypt.org/acme/finalize/124272533/11192376274' [Mon 19 Jul 2021 05:51:40 PM CEST] url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/14959992950' [Mon 19 Jul 2021 05:51:40 PM CEST] payload [Mon 19 Jul 2021 05:51:40 PM CEST] POST [Mon 19 Jul 2021 05:51:40 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/authz-v3/14959992950' [Mon 19 Jul 2021 05:51:40 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:40 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:40 PM CEST] code='200' [Mon 19 Jul 2021 05:51:40 PM CEST] d='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:40 PM CEST] Getting webroot for domain='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:40 PM CEST] _w='/usr/local/ispconfig/interface/acme' [Mon 19 Jul 2021 05:51:40 PM CEST] _currentRoot='/usr/local/ispconfig/interface/acme' [Mon 19 Jul 2021 05:51:40 PM CEST] entry='"type":"http-01","status":"pending","url":"https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA","token":"Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U"' [Mon 19 Jul 2021 05:51:40 PM CEST] token='Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U' [Mon 19 Jul 2021 05:51:40 PM CEST] uri='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:40 PM CEST] keyauthorization='Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U.0STKIhSzQpWD-NJb2ZuSv9JIfEi0ASkA21dZnpiDHtk' [Mon 19 Jul 2021 05:51:40 PM CEST] dvlist='nlhb52619ds.vibel.be#Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U.0STKIhSzQpWD-NJb2ZuSv9JIfEi0ASkA21dZnpiDHtk#https://acme-v02.api.letsencrypt.or...A#http-01#/usr/local/ispconfig/interface/acme' [Mon 19 Jul 2021 05:51:40 PM CEST] d [Mon 19 Jul 2021 05:51:40 PM CEST] vlist='nlhb52619ds.vibel.be#Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U.0STKIhSzQpWD-NJb2ZuSv9JIfEi0ASkA21dZnpiDHtk#https://acme-v02.api.letsencrypt.or...A#http-01#/usr/local/ispconfig/interface/acme,' [Mon 19 Jul 2021 05:51:41 PM CEST] d='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:41 PM CEST] ok, let's start to verify [Mon 19 Jul 2021 05:51:41 PM CEST] Verifying: nlhb52619ds.vibel.be [Mon 19 Jul 2021 05:51:41 PM CEST] d='nlhb52619ds.vibel.be' [Mon 19 Jul 2021 05:51:41 PM CEST] keyauthorization='Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U.0STKIhSzQpWD-NJb2ZuSv9JIfEi0ASkA21dZnpiDHtk' [Mon 19 Jul 2021 05:51:41 PM CEST] uri='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:41 PM CEST] _currentRoot='/usr/local/ispconfig/interface/acme' [Mon 19 Jul 2021 05:51:41 PM CEST] wellknown_path='/usr/local/ispconfig/interface/acme/.well-known/acme-challenge' [Mon 19 Jul 2021 05:51:41 PM CEST] writing token:Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U to /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U [Mon 19 Jul 2021 05:51:41 PM CEST] Changing owner/group of .well-known to ispconfig:ispconfig [Mon 19 Jul 2021 05:51:41 PM CEST] url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:41 PM CEST] payload='{}' [Mon 19 Jul 2021 05:51:41 PM CEST] POST [Mon 19 Jul 2021 05:51:41 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:41 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:41 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:41 PM CEST] code='200' [Mon 19 Jul 2021 05:51:41 PM CEST] trigger validation code: 200 [Mon 19 Jul 2021 05:51:41 PM CEST] sleep 2 secs to verify [Mon 19 Jul 2021 05:51:43 PM CEST] checking [Mon 19 Jul 2021 05:51:43 PM CEST] url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:43 PM CEST] payload [Mon 19 Jul 2021 05:51:43 PM CEST] POST [Mon 19 Jul 2021 05:51:43 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:43 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:44 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:44 PM CEST] code='200' [Mon 19 Jul 2021 05:51:44 PM CEST] nlhb52619ds.vibel.be:Verify error:Fetching http://nlhb52619ds.vibel.be/.well-known/acme-challenge/Ctj-jgnb4-xBbhUMxdn_xANhovMbSO9Zk-z50YijG2U: Connection refused [Mon 19 Jul 2021 05:51:44 PM CEST] pid [Mon 19 Jul 2021 05:51:44 PM CEST] No need to restore nginx, skip. [Mon 19 Jul 2021 05:51:44 PM CEST] _clearupdns [Mon 19 Jul 2021 05:51:44 PM CEST] dns_entries [Mon 19 Jul 2021 05:51:44 PM CEST] skip dns. [Mon 19 Jul 2021 05:51:44 PM CEST] _on_issue_err [Mon 19 Jul 2021 05:51:44 PM CEST] Please check log file for more details: /var/log/ispconfig/acme.log [Mon 19 Jul 2021 05:51:44 PM CEST] url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:44 PM CEST] payload='{}' [Mon 19 Jul 2021 05:51:44 PM CEST] POST [Mon 19 Jul 2021 05:51:44 PM CEST] _post_url='https://acme-v02.api.letsencrypt.org/acme/chall-v3/14959992950/bUTEmA' [Mon 19 Jul 2021 05:51:44 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:51:45 PM CEST] _ret='0' [Mon 19 Jul 2021 05:51:45 PM CEST] code='400' [Mon 19 Jul 2021 05:52:44 PM CEST] Running cmd: upgrade [Mon 19 Jul 2021 05:52:44 PM CEST] Using config home:/root/.acme.sh [Mon 19 Jul 2021 05:52:44 PM CEST] default_acme_server='https://acme-v02.api.letsencrypt.org/directory' [Mon 19 Jul 2021 05:52:44 PM CEST] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory' [Mon 19 Jul 2021 05:52:44 PM CEST] GET [Mon 19 Jul 2021 05:52:44 PM CEST] url='https://api.github.com/repos/acmesh-official/acme.sh/git/refs/heads/master' [Mon 19 Jul 2021 05:52:44 PM CEST] timeout= [Mon 19 Jul 2021 05:52:44 PM CEST] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g ' [Mon 19 Jul 2021 05:52:44 PM CEST] ret='0' [Mon 19 Jul 2021 05:52:44 PM CEST] Already uptodate! [Mon 19 Jul 2021 05:52:44 PM CEST] Upgrade success! [Mon 19 Jul 2021 05:52:44 PM CEST] Running cmd: setdefaultca [Mon 19 Jul 2021 05:52:44 PM CEST] Changed default CA to: https://acme-v02.api.letsencrypt.org/directory
Apache2 crashes daily as a result of this issue. The only way to restart Apache2 is to redo the procedure (commenting out the SSL references in apps.vhost & ipconfig.vhost and restarting Apache2) Surelly there must be a better way to cure this ?!
I can assure you that no server settings have been changed since the "installing the perfect debian 10 server" and ispconfig 3 was installed. The issue therefore must be related to a cron job by ispconfig3.2, and I will buy you a beer if i'm proven wrong. I did in fact request business support through the offical channel, because it's essential.
