recovering from disaster....

@craig baker
To restore the system certificate and all its renewal behaviour do the following:
(assuming acme.sh is being used for certificate issue and renewal)

First get the webserver running, at least listening on port 80.
So if apps-vhost and/or ispconfig-vhost are the problem because of the corrupted system certificate delete their symlinks in /etc/apache2/sites-enabled and restart apache.

Code:

rm -f etc/apache2/sites-enabled/000-apps.vhost
rm -f etc/apache2/sites-enabled/000-ispconfig.vhost
systemctl restart apache2

Do the same with other websites also causing apache not to start.

Delete the system certificate in /root/.acme.sh

Code:

rm -rf /root/.acme.sh/<system hostname>

Now to recreate the system certificate again do the following

Code:

acme.sh --issue -d <system hostname> -w "/usr/local/ispconfig/interface/acme" --always-force-new-domain-key --keylength 4096 --key-file "/usr/local/ispconfig/interface/ssl/ispserver.key" --fullchain-file "/usr/local/ispconfig/interface/ssl/ispserver.crt" --renew-hook "letsencrypt_renew_hook.sh"

If you want to add extra hostnames in the certificate then add extra -d parameters in the command

Code:

... -d <system hostname> -d <extra host A> -d <extra host B> ...

Also if you want an ecdsa certificate instead of a rsa certificate you can replace keylength

Code:

... --keylength ec-256 ...

Issueing the certificate now shouldn't be any problem as long as apache is running and listening on port 80.
When completed restore apps-vhost and ispconfig-vhost

Code:

ln -s /etc/apache2/sites-available/apps.vhost /etc/apache2/sites-enabled/000-apps.vhost
ln -s /etc/apache2/sites-available/ispconfig.vhost /etc/apache2/sites-enabled/000-ispconfig.vhost
systemctl restart apache2

When apache has restarted successfully and you had disabled other websites too by deleting their symlink you can now restore them the same way one by one.

Last step is to do a force renew, since the renewal hook is only triggered on renewal of the certificate

Code:

acme.sh --renew --force -d <system hostname>

Now your system certificate and its renewal behaviour is completely restored to default without any need of a website and replacing certificates by a symlink and/or altering config of services using that certificate by default.

 

I would always and ever prefer just adding the extra hostnames directly into the certificate's conf file and do a force renew when the system's hostname is involved in the certificate.
As I said before is the tutorial perfectly fine to replace the certificate in postfix and dovecot with one NOT containing the system's hostname.
Just leave out the symlink part to /usr/local/ispconfig/interface/ssl.

 

Sorry, can't help with that
It's been a long time since I last used certbot.
And back then never looked into how the system certificate gets created.
So not a clue how exactly the certificate is installed in /usr/local/ispconfig/interface/ssl and how the renewal hook is done.

Maybe a good time for @craig baker to replace certbot with acme.sh since things aren't working properly already
There is a tutorial here somewhere about howto.

 

but remkoh I'm MIGRATING from an oldersite using certbot. hence I had to add --use-certbot!
so..... certbot instructions?

maybe I'm missing something but I get to ispconfig via the hostname:8080
so ns11.cdbsystems.com:8080
but that requires a vhost somewhere...

 

thats a negatory! (nein) cannot connect to https://ns11.cdbsystems.com:8080

 

created the self signed cert per your instructions. restarted apache2. still no joy.

we did:
cd /usr/local/ispconfig/interface/ssl
openssl genrsa -des3 -out ispserver.key 4096
openssl req -new -key ispserver.key -out ispserver.csr
openssl x509 -req -days 3650 -in ispserver.csr \
-signkey ispserver.key -out ispserver.crt
openssl rsa -in ispserver.key -out ispserver.key.insecure
mv ispserver.key ispserver.key.secure
mv ispserver.key.insecure ispserver.key

except got red of the -des3 as that demains a passphrase as others have pointed.
cannot connect to https://ns11.cdbsystems.com:8080

 

Sorry, can't help you with that.
I tried digesting the certbot equivalent from /usr/local/ispconfig/server/lib/classes/letsencrypt.inc.php combined with certbot's available parameters.
But I'm no professional programmer. It's just a bit to gibberish to me to get to the complete result, only partial.

 

ok apache2 running.
it was NOT listening on 8080 via netstat.
I had moved the ispconfig.vhost to /root as it prevented apache2 from starting.
moved it back
apache2 does now start -
and its listening on 8080

however https://ns11.cdbsystems.com:8080

still refuses to connect due to security issue (self signed cert)
sigh.

 

sorry till was trying to get apache2 to run - other websites were all down when ispconfig.vhost was there...
so its back in place. firefox and chrome refuse to connect as its self signed - but opera will connect, so I'm in ispconfig3...
i can log in without ssl. tried ispconfig_update --force told it make new ssl. but it did not.
however at least I'm able to do the migration to the ns11 server - I guess the ns11 ssl will have to wait!

 

A good reminder not to create web site with server FQDN hostname, but I think you should revoke the LE SSL certs before deleting its folder, so it is cleanly removed.

And in my mind, except to manually revoke and delete existing LE SSL certs, which I did also advise earlier, ispconfig force update already covers all that, so I don't know why one would go to all that hassle for the server FQDN hostname with manual command, certbot or acme.sh, but of course you can.

 

its certbot not acme

 

True, though they after max 3 once they've expired anyway.
Revoking is mainly for leaked keys and things like that. If your key is safe it's not a necessity.

If one wants extra hostnames in their certificate they have not much of a choise.
Doing it immediately with one command (or two to run the renewal hook afterwards, though it can be combined) or do an ispc force update and afterwards add extra hostnames with an acme force renew, the endresult is the same.

 

Nope. It is still not clean without revoking it first as you will still at least get all expiration warning emails. LE has no ways of knowing whether you are still using them or not, so will keep sending them, until you revoked them in a proper manner. Yes, they are ignorable but still annoying. Besides, it is the right and safest way.

As you said yourself, one may extend his server FQDN hostame to others by doing that afterwards, so not necessarily before, though possible, and extending them with extend command thereafter is the best, and to note that this extend command is also a default command use in ISPConfig to extend a web site domain to its subdomain.

Why using ISPConfig installer / updater to issue LE SSL certs? Because server FQDN hostname will has the needed LE SSL certs to readily operate with them, and all other services linked to them, with all needed hooks for future renewal, as obviously, first and foremost, complete server setup always come first.

Also, there are risks of other FQDN add up failing for any reasons, so that won't affect the already obtained certs, and one can take time to fix it thereafter, so one step at a time will do best.

But again, of course, as advanced and seasoned admin, one can also do all of them, together, manually, step by step too, no doubt about it.

 

Yes LE can't know, though they have stopped sending warnings about expirations for quite some time now. Hard to ignore something that isn't send.

Not disputing that. Just saying it is not a necessity as long as your key hasn't become public knowledge.
And even then, if the old key is deleted then no service can use it anymore. At least not on your own server.
When somebody is able to use your old key and domainname on a totally different server you have a whole other problem.