My ispconfig control panel is unreachable, due to its SSL cert having expired. This was working until recently, with automatic renewals happening ok. Also I have another similar server, for which this sipconfig control panel problem does not exist. I looked at https://www.howtoforge.com/tutorial/securing-ispconfig-3-with-a-free-lets-encrypt-ssl-certificate/, but since that turotial was for ispconfig pre-3.2, and I have 3.2.9p1, I did not use it. Per the "read before posting" post, I have included the output of the test script as suggested in a code block below. Also I see from the symbolic links that /usr/local/ispconfig/interface/ssl/ispserver.crt -> /etc/letsencrypt/live/xxxx-server-name/fullchain.pem which points ultimately to a file created on the expiration date of the expired SSL cert. This is not the case on my other server, which points to a file created on the creation date of the currently valid SSL cert for that server's ispconfig control panel. Both servers are running Ubuntu 20.04.5 LTS. The Linux kernel of the server with the problem is GNU/Linux 5.4.0-135-generic x86_64, while the server without this problem is running GNU/Linux 6.0.2-x86_64-linode157 x86_64. However, the problem server's ispconfig SSL for its control panel worked fine until recently. Thanks for suggestions to get SSL working, renewed. I tried update: ispconfig_update.sh --force with no effect. Code: # cat htf_report.txt ##### SERVER ##### IP-address (as per hostname): ***.***.***.*** [WARN] could not determine server's ip address by ifconfig [INFO] OS version is Ubuntu 20.04.5 LTS [INFO] uptime: 14:35:20 up 13 days, 20:38, 3 users, load average: 0.10, 0.03, 0.05 [INFO] memory: total used free shared buff/cache available Mem: 3.8Gi 2.1Gi 141Mi 265Mi 1.6Gi 1.2Gi Swap: 2.0Gi 1.2Gi 840Mi [INFO] ISPConfig is installed. ##### ISPCONFIG ##### ISPConfig version is 3.2.9p1 ##### VERSION CHECK ##### [INFO] php (cli) version is 7.4.33 [INFO] php-cgi (used for cgi php in default vhost!) is version 7.4.33 ##### PORT CHECK ##### ##### MAIL SERVER CHECK ##### ##### RUNNING SERVER PROCESSES ##### [INFO] I found the following web server(s): Apache 2 (PID 3193259) [INFO] I found the following mail server(s): Postfix (PID 3193095) [INFO] I found the following pop3 server(s): Dovecot (PID 3193229) [INFO] I found the following imap server(s): Dovecot (PID 3193229) [INFO] I found the following ftp server(s): PureFTP (PID 3193287) ##### LISTENING PORTS ##### (only () Local (Address) [localhost]:953 (3193304/named) [anywhere]:25 (3193095/master) [anywhere]:993 (3193229/dovecot) [anywhere]:995 (3193229/dovecot) [localhost]:10023 (1189/postgrey) [localhost]:10024 (3193216/amavisd-new) [anywhere]:47784 (1574/docker-proxy) [localhost]:10025 (3193095/master) [localhost]:10026 (3193216/amavisd-new) [localhost]:10027 (3193095/master) [anywhere]:587 (3193095/master) [localhost]:11211 (740/memcached) [anywhere]:110 (3193229/dovecot) [anywhere]:143 (3193229/dovecot) [anywhere]:10000 (1683/perl) [anywhere]:465 (3193095/master) ***.***.***.***:53 (3193304/named) ***.***.***.***:53 (3193304/named) [localhost]:53 (3193304/named) [anywhere]:21 (3193287/pure-ftpd) ***.***.***.***:53 (673/systemd-resolve) [anywhere]:22 (921/sshd:) *:*:*:*::*:25 (3193095/master) *:*:*:*::*:953 (3193304/named) *:*:*:*::*:443 (3193259/apache2) *:*:*:*::*:993 (3193229/dovecot) *:*:*:*::*:995 (3193229/dovecot) *:*:*:*::*:10023 (1189/postgrey) *:*:*:*::*:10024 (3193216/amavisd-new) *:*:*:*::*:47784 (1594/docker-proxy) *:*:*:*::*:10026 (3193216/amavisd-new) *:*:*:*::*:3306 (3192827/mariadbd) *:*:*:*::*:587 (3193095/master) [localhost]10 (3193229/dovecot) [localhost]43 (3193229/dovecot) *:*:*:*::*:8080 (3193259/apache2) *:*:*:*::*:80 (3193259/apache2) [localhost]0000 (1683/perl) *:*:*:*::*:8081 (3193259/apache2) *:*:*:*::*:465 (3193095/master) *:*:*:*::*b848:51ff:feb7:53 (3193304/named) *:*:*:*::*42:16ff:fe9a:7:53 (3193304/named) *:*:*:*::*f03c:91ff:fe70:53 (3193304/named) *:*:*:*::*f03c:91ff:53 (3193304/named) *:*:*:*::*:53 (3193304/named) *:*:*:*::*:21 (3193287/pure-ftpd) *:*:*:*::*:4949 (1148/perl) *:*:*:*::*:22 (921/sshd:) ##### IPTABLES ##### Chain INPUT (policy DROP) target prot opt source destination f2b-pure-ftpd tcp -- [anywhere]/0 [anywhere]/0 multiport dports 21 f2b-sshd tcp -- [anywhere]/0 [anywhere]/0 multiport dports 22 ufw-before-logging-input all -- [anywhere]/0 [anywhere]/0 ufw-before-input all -- [anywhere]/0 [anywhere]/0 ufw-after-input all -- [anywhere]/0 [anywhere]/0 ufw-after-logging-input all -- [anywhere]/0 [anywhere]/0 ufw-reject-input all -- [anywhere]/0 [anywhere]/0 ufw-track-input all -- [anywhere]/0 [anywhere]/0 Chain FORWARD (policy DROP) target prot opt source destination DOCKER-USER all -- [anywhere]/0 [anywhere]/0 DOCKER-ISOLATION-STAGE-1 all -- [anywhere]/0 [anywhere]/0 ACCEPT all -- [anywhere]/0 [anywhere]/0 ctstate RELATED,ESTABLISHED DOCKER all -- [anywhere]/0 [anywhere]/0 ACCEPT all -- [anywhere]/0 [anywhere]/0 ACCEPT all -- [anywhere]/0 [anywhere]/0 ufw-before-logging-forward all -- [anywhere]/0 [anywhere]/0 ufw-before-forward all -- [anywhere]/0 [anywhere]/0 ufw-after-forward all -- [anywhere]/0 [anywhere]/0 ufw-after-logging-forward all -- [anywhere]/0 [anywhere]/0 ufw-reject-forward all -- [anywhere]/0 [anywhere]/0 ufw-track-forward all -- [anywhere]/0 [anywhere]/0 Chain OUTPUT (policy ACCEPT) target prot opt source destination ufw-before-logging-output all -- [anywhere]/0 [anywhere]/0 ufw-before-output all -- [anywhere]/0 [anywhere]/0 ufw-after-output all -- [anywhere]/0 [anywhere]/0 ufw-after-logging-output all -- [anywhere]/0 [anywhere]/0 ufw-reject-output all -- [anywhere]/0 [anywhere]/0 ufw-track-output all -- [anywhere]/0 [anywhere]/0 Chain DOCKER (1 references) target prot opt source destination ACCEPT tcp -- [anywhere]/0 ***.***.***.*** tcp dpt:80 Chain DOCKER-ISOLATION-STAGE-1 (1 references) target prot opt source destination DOCKER-ISOLATION-STAGE-2 all -- [anywhere]/0 [anywhere]/0 RETURN all -- [anywhere]/0 [anywhere]/0 Chain DOCKER-ISOLATION-STAGE-2 (1 references) target prot opt source destination DROP all -- [anywhere]/0 [anywhere]/0 RETURN all -- [anywhere]/0 [anywhere]/0 Chain DOCKER-USER (1 references) target prot opt source destination RETURN all -- [anywhere]/0 [anywhere]/0 Chain f2b-pure-ftpd (1 references) target prot opt source destination RETURN all -- [anywhere]/0 [anywhere]/0 Chain f2b-sshd (1 references) target prot opt source destination REJECT all -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable RETURN all -- [anywhere]/0 [anywhere]/0 Chain ufw-after-forward (1 references) target prot opt source destination Chain ufw-after-input (1 references) target prot opt source destination ufw-skip-to-policy-input udp -- [anywhere]/0 [anywhere]/0 udp dpt:137 ufw-skip-to-policy-input udp -- [anywhere]/0 [anywhere]/0 udp dpt:138 ufw-skip-to-policy-input tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:139 ufw-skip-to-policy-input tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:445 ufw-skip-to-policy-input udp -- [anywhere]/0 [anywhere]/0 udp dpt:67 ufw-skip-to-policy-input udp -- [anywhere]/0 [anywhere]/0 udp dpt:68 ufw-skip-to-policy-input all -- [anywhere]/0 [anywhere]/0 ADDRTYPE match dst-type BROADCAST Chain ufw-after-logging-forward (1 references) target prot opt source destination LOG all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] " Chain ufw-after-logging-input (1 references) target prot opt source destination LOG all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] " Chain ufw-after-logging-output (1 references) target prot opt source destination Chain ufw-after-output (1 references) target prot opt source destination Chain ufw-before-forward (1 references) target prot opt source destination ACCEPT all -- [anywhere]/0 [anywhere]/0 ctstate RELATED,ESTABLISHED ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 3 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 11 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 12 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 8 ufw-user-forward all -- [anywhere]/0 [anywhere]/0 Chain ufw-before-input (1 references) target prot opt source destination ACCEPT all -- [anywhere]/0 [anywhere]/0 ACCEPT all -- [anywhere]/0 [anywhere]/0 ctstate RELATED,ESTABLISHED ufw-logging-deny all -- [anywhere]/0 [anywhere]/0 ctstate INVALID DROP all -- [anywhere]/0 [anywhere]/0 ctstate INVALID ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 3 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 11 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 12 ACCEPT icmp -- [anywhere]/0 [anywhere]/0 icmptype 8 ACCEPT udp -- [anywhere]/0 [anywhere]/0 udp spt:67 dpt:68 ufw-not-local all -- [anywhere]/0 [anywhere]/0 ACCEPT udp -- [anywhere]/0 ***.***.***.*** udp dpt:5353 ACCEPT udp -- [anywhere]/0 ***.***.***.*** udp dpt:1900 ufw-user-input all -- [anywhere]/0 [anywhere]/0 Chain ufw-before-logging-forward (1 references) target prot opt source destination Chain ufw-before-logging-input (1 references) target prot opt source destination Chain ufw-before-logging-output (1 references) target prot opt source destination Chain ufw-before-output (1 references) target prot opt source destination ACCEPT all -- [anywhere]/0 [anywhere]/0 ACCEPT all -- [anywhere]/0 [anywhere]/0 ctstate RELATED,ESTABLISHED ufw-user-output all -- [anywhere]/0 [anywhere]/0 Chain ufw-logging-allow (0 references) target prot opt source destination LOG all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW ALLOW] " Chain ufw-logging-deny (2 references) target prot opt source destination RETURN all -- [anywhere]/0 [anywhere]/0 ctstate INVALID limit: avg 3/min burst 10 LOG all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 10 LOG flags 0 level 4 prefix "[UFW BLOCK] " Chain ufw-not-local (1 references) target prot opt source destination RETURN all -- [anywhere]/0 [anywhere]/0 ADDRTYPE match dst-type LOCAL RETURN all -- [anywhere]/0 [anywhere]/0 ADDRTYPE match dst-type MULTICAST RETURN all -- [anywhere]/0 [anywhere]/0 ADDRTYPE match dst-type BROADCAST ufw-logging-deny all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 10 DROP all -- [anywhere]/0 [anywhere]/0 Chain ufw-reject-forward (1 references) target prot opt source destination Chain ufw-reject-input (1 references) target prot opt source destination Chain ufw-reject-output (1 references) target prot opt source destination Chain ufw-skip-to-policy-forward (0 references) target prot opt source destination DROP all -- [anywhere]/0 [anywhere]/0 Chain ufw-skip-to-policy-input (7 references) target prot opt source destination DROP all -- [anywhere]/0 [anywhere]/0 Chain ufw-skip-to-policy-output (0 references) target prot opt source destination ACCEPT all -- [anywhere]/0 [anywhere]/0 Chain ufw-track-forward (1 references) target prot opt source destination Chain ufw-track-input (1 references) target prot opt source destination Chain ufw-track-output (1 references) target prot opt source destination ACCEPT tcp -- [anywhere]/0 [anywhere]/0 ctstate NEW ACCEPT udp -- [anywhere]/0 [anywhere]/0 ctstate NEW Chain ufw-user-forward (1 references) target prot opt source destination Chain ufw-user-input (1 references) target prot opt source destination ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:20 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:21 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:22 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:53 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:80 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:110 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:143 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:443 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:465 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:587 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:993 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:995 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:3306 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:8080 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:8081 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:10000 ACCEPT udp -- [anywhere]/0 [anywhere]/0 udp dpt:53 ACCEPT udp -- [anywhere]/0 [anywhere]/0 udp dpt:3306 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:47784 ACCEPT tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:25 Chain ufw-user-limit (0 references) target prot opt source destination LOG all -- [anywhere]/0 [anywhere]/0 limit: avg 3/min burst 5 LOG flags 0 level 4 prefix "[UFW LIMIT BLOCK] " REJECT all -- [anywhere]/0 [anywhere]/0 reject-with icmp-port-unreachable Chain ufw-user-limit-accept (0 references) target prot opt source destination ACCEPT all -- [anywhere]/0 [anywhere]/0 Chain ufw-user-logging-forward (0 references) target prot opt source destination Chain ufw-user-logging-input (0 references) target prot opt source destination Chain ufw-user-logging-output (0 references) target prot opt source destination Chain ufw-user-output (1 references) target prot opt source destination ##### LET'S ENCRYPT ##### Certbot is installed in /usr/bin/letsencrypt
There was a file, ispserver.crt, in /usr/local/ispconfig/interface/ssl/ created on the date of the ispconfig update. However, in that dir, ispserver.crt linked to a file created on the creation date of the expired SSL cert. On the server with the valid SSL cert, the corresponding link went to a file created on the creation date of the valid SSL cert. On the server with the expired cert, there was a file created on the date of the update, however, in the same dir: ispserver.pem. The file ispserver.pem on the server with the valid SSL cert was created several years ago. The same file on the server with the SSL expired was created on the date of the ispconfig update, earlier this month (March 2023). It seems the new cert is being created in the dir /usr/local/ispconfig/interface/ssl/ on the server with the expired SSL, while it is being created in another directory, and being linked to from /usr/local/ispconfig/interface/ssl/ in the server with the valid SSL.
1. Check whether your certbot was installed via apt or snapd, if via apt, uninstall thoroughly and re-install via snapd, as instructed in certbot official website. 2. Force update ISPConfig and choose create SSL during that process; see if that fix your problem. 3. Otherwise, check your LE logs and follows LE FAQ to troubleshoot your problem.
Which Let's encrypt client to do youse? SSL is handled differently depending on which LE client you are using, this means you can not compare different servers with different LE clients. Recent systems typically use acme.sh, old systems typically use certbot. So the first step is to find out which let's encrypt client you are using on the system with the expired cert.
Thanks, First, I backed up my server. Then, following https://certbot.eff.org/instructions?ws=apache&os=ubuntufocal, I did the following: Code: root@cosmo:~# apt install snapd ... root@cosmo:~# apt-get remove certbot ... The following packages will be REMOVED: certbot ... After this operation, 51.2 kB disk space will be freed. ... Do you want to continue? [Y/n] ... Removing certbot (0.40.0-1ubuntu0.1) ... Processing triggers for man-db (2.9.1-1) ... root@cosmo:~# ... root@cosmo:~# sudo dpkg -l *certbot* Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-======================-=================-============-=====================> rc certbot 0.40.0-1ubuntu0.1 all automatically configu> un python-certbot-apache <none> <none> (no description avail> un python-certbot-doc <none> <none> (no description avail> un python-certbot-nginx <none> <none> (no description avail> ii python3-certbot 0.40.0-1ubuntu0.1 all main library for cert> un python3-certbot-apache <none> <none> (no description avail> un python3-certbot-nginx <none> <none> (no description avail> root@cosmo:~# root@cosmo:~# snap install --classic certbot certbot 2.4.0 from Certbot Project (certbot-eff✓) installed root@cosmo:~# which certbot /snap/bin/certbot root@cosmo:~# ln -s /snap/bin/certbot /usr/bin/certbot The ispconfig control panel still reports an invalid cert. Would I need to setup some more symbolic links, remove some of the packages listed above, or try to force re-generation of the cert? How would forcing regeneration of the cert best be done? This cert is for ispconfig itself, other websites' certs on the server were ok / valid. Thanks for any suggestions!
Haven't you try step #8 yet to test and thereafter run the renewal command i.e. without dry-run param to renew the certs? Of course you can wait default daily renewal which is run somewhere midnight or thereafter.
Code: rc certbot 0.40.0-1ubuntu0.1 all automatically configu> Host still has configuration files of the original certbot installed. You should remove them with apt-get purge certbot, although I do not know if that messes up the certbot from snap.
Thanks guys, I appreciate the help! I may just manually upload a self-signed cert and set the links to it - at least for now (till auto-renewal time). I am thinking ispserver.crt and ispserver.key are the two links to set for this, from the list below. Code: /usr/local/ispconfig/interface/ssl# l -1 dhparam4096.pem* empty.dir* ispserver.crt@ ispserver.crt-190501111337.bak* ispserver.crt.bak-2023-02-20@ ispserver.crt.test-2023-02-20@ ispserver.csr* ispserver.key@ ispserver.key-190501111337.bak* ispserver.key.secure* ispserver.pem* ispserver.pem-20230220173240.bak* ispserver.pem-20230304004620.bak*
After another ispconfig_updqate --force I found a newly created ispserver.pem but not a newly created .key file, in /usr/local/ispconfig/interface/ssl . ispserver.crt was there but still linked to the same, expired, .pem file. The ispserver.key links to the same key file as before the forced ispconfig update, created on the same date as the expired cert. On another similar server, the key and fullchain files in corresponding locations were created on the creation date of the valid cert. Also on the other similar server, the ispserver.pem in /usr/local/ispconfig/interface/ssl is much older. So, on the server with the problem, there is a newly created ispserver.pem but not the newly created file linked to by ispserver.crt. I suspect the newly created ispserver.pem contains the correct cert, but wonder where is the corresonding key? It is not in the key location as on the other server with the cert that works. Can anyone tell me where to find the .key file that goes with this .pem file? (Note: looked at https://eff-certbot.readthedocs.io/en/stable/using.html#where-are-my-certificates and the README in /etc/letsencrypt/live/server-domain-name and the cert and key files are NOT updated in the problem server, though they are updated in the other server that has a valid cert.) Perhaps manually linking to the right files will work (till the next auto-renew) as a temporary kluge, work-around. Thanks!
FYI, this is an update: I was able to login with the invalid (expired) cert, by manually editing the vhost file and commenting out the HSTS section. I plan to try a few other things. By the way, removing the certbot config did seem to impact other sites, so I reverted that. I may try manually emptying the dir containing the ispconfig certs only, and then re-updating ispconfig with --force.
An update: FYI, this did not work. However, manually removing HSTS from the .vhost file is still one workaround.
FYI, in case anyone has a similar problem - the following worked. I just followed the suggestion on https://stackoverflow.com/questions...-of-a-lets-encrypt-certificate-in-ispconfig-3 which worked. Briefly, 1- disable Let's Encrypt (though not ssl) for the site in ISPConfig 2- remove (or rename) the following: (they apparently will get re-created in step 3) /etc/letsencrypt/archive/site-name/ /etc/letsencrypt/live/site-name/ /etc/letsencrypt/renewal/site-name/ 3- re-enable Let's Encrypt in ISPConfig It worked for me!
