recovering from disaster....

It cannot.
First you need to get apache running so http://servername/.well-known/acme-challenge/ can be reached by LE servers.
That equals folder /usr/local/ispconfig/interface/acme/.well-known/acme-challenge where LE will create a validation file during requesting/issueing a certificate.
To get apache running you should delete symlinks in /etc/apache2/sites-enabled of websites that prevent apache from running.

 

problem is that I cant get apache2 running without the ns11 certs. (the ones in /usr/local/ispconfig).
however the key and cert for ns11 do NOT match.
[Wed Nov 12 11:39:54.381364 2025] [ssl:emerg] [pid 2895189:tid 2895189] AH02565: Certificate and private key ns11.cdbsystems.com:8081:0 from /usr/local/ispconfig/interface/ssl/ispserver.crt and /usr/local/ispconfig/interface/ssl/ispserver.key do not match

so apache2 cannot run.

ispconfig_update.sh cannot generate new certs because apache2 is down.

seems like a catch22 to me. now I have deleted the ispconfig.vhost files from sites-enabled, but update_config.sh --force of course cant recreate it - because apache2 cant run.

now I grepped /etc/apache2/sites-available for /usr/local/ispconfig and they were in apps.vhost.
I moved those to /root and now apache2 loads, and I'm rerunning ispconfig_update.sh --force

will it recreate the apps.vhost when its missing?

seems it does. but apache2 still cannot load.
I mved all from /usr/locl/interface/ssl/* to a /root/folder.
then apache2 could load. and ispconfig_update.sh --force seems to have recreated the files.
maybe things might now be working??

sigh... nope.
ns11.cdbsystems.com:8080 does not connect at all. guess thats because there is no ispconfig.vhost?

round and round I go. I move the /root/ ispconfig.vhost back to /etc/apache2/sites-* then apache2 again cannot load as the cert and key dont match.

sigh. might be time to wipe server start over? surely these is a way out of this?

 

When you update, there is always a backup done, so deleting ispconfig vhost is not really big deal, you can always restore from backup.

Your problem is the certs you specified therein, which I was trying to help, but I am not so sure how did you do things there, when you force update, but we normally do backup, re-calibrate all services, opt to get ssl certs, and they normally are just fine.

From my side, your server apache2 seems to work fine in without port 8080 and 8081, and on port 8081 without the certs, so the easiest out for now, if you really want the ISPConfig gui temporarily, just restore ispconfig vhost, remove the ssl line and restart apache2, but may be you do not just want that.

It is your broken old server you are trying to restore, your own choice whether to fix things, or wipe server and start over, from scratch or backups. To me, in a test vm, before production, we can do that again and again and again, until we are satisfied that all are working good.

 

the actual problem as apache2 keeps saying is the the key and cert in /etc/letsencrypt/whatever/ns11.cdbsystems.com_ecc folder do not match. and these are symlinked from ispconfig.vhost.
how can I force the 2 certs to be recreated and match? (they are both dated oct12 so the ispconfig_update.sh --force and saying 'redo 8080 ssl' clearly did NOT work)
if I could do that everything should work!!

 

ok still on the recoverying from disaster. I have received back the drive from the cleanroom. $8000 sheesh.
But they recovered a raid 6 array of 8 10tb drives. quite a lot of work! as 5 drives did NOT spin up after the disaster.
Now, I need to combine the emails from dovecot boxes, before and after the disaster.
Can I simply copy over the maildir folder for one of the users in the recovered domain, and delete the dovecot cache file? is that sufficient to merge the emails, old and new??
or do I need to copy over the WHOLE dovecot folder for that domain, after maybe using imapsync to copy the 'new mails' (only 1 month worth!) to another box temporarily?

and I'm copying all the /var/lib/mysql folder to another deb12 server. I'm hoping mysqldump will work to create the sql file that then I can import with phpmyadmin after copying the /var/www/mydeadcomain contents to a mydeaddomain folder that I create, then creating the database (in wp-config.php), and doing an sql import with the above file with phpmyadmin? that SHOULD bring up a dead website, shouldnt it (they are all wordpress).
any thoughts?

 

thats fine, the bases are really NOT large. I'm just trying to udnerstand if I copy the /var/lib/mysql folder (including the databases) to a new deb12 server, should mysqldump work on them? so I somehow need to register them?

and my dovecot problem? what do you think of that?

 

ENLIGHTENMENT -
upon zipping up /var/lib/mysql and unpacking it on a deb12 system, I could NOT start mysql/mariadb.
It kept giving 'InnoDB' related errors, corrupt database errors.
I poked around and (at its suggestion) added
innodb_force_recovery = 1
into /etc/mysql/mariadb.conf.d/50-server.cnf
under [mysql]
and mysql could start up! I'm dumping the bases now with mysqldump!

first old website RESTORED -
copied data from /var/www/mydomain.com on crashed drive to /var/www/mydomain.com
created a new sql database c0mydomainnew
alter /var/www/mydomain.com/web/wp-coinfig.php to replace databasename with c0mydomainnew
mysql -u root -p c0mydomainnew < mysqlfilefromdumponother server

I note I have to delete a line at the top of each sql file:
/*M!999999\- enable the sandbox mode */
else it give an sql error '\- not valid command!

wordpress is happy - site is up!

 

GEEZ!!!! I am an idiot. I followed your directions till to create a self-signed cert, but did it on NS10 NOT ns11!!!
and ispconfig_update.sh --force does NOT fix it.
from the letsencrypt log:
2025-11-15 18:45:12,542:INFO:certbot.ocsp:Cannot extract OCSP URI from /etc/letsencrypt/archive/ns10.cdbsystems.com/cert31.pem
2025-11-15 18:45:12,547:INFO:certbot._internal.renewal:Cert not yet due for renewal
2025-11-15 18:45:12,547:INFO:certbot._internal.main:Keeping the existing certificate
how do I fix please?

from /usr/local/ispconfig/interface/ssl:
[root@ns10 ssl]# ls -al
total 36
drwxr-x--- 2 root root 254 Nov 15 18:45 .
drwxr-x--- 9 ispconfig ispconfig 121 Oct 1 2020 ..
-rwxr-x--- 1 root root 768 Nov 15 18:45 dhparam4096.pem
-rwxr-x--- 1 root root 430 Oct 1 2020 DOIT.bat
-rwxr-x--- 1 root root 45 Nov 15 18:45 empty.dir
lrwxrwxrwx 1 root root 55 Nov 15 18:45 ispserver.crt -> /etc/letsencrypt/live/ns10.cdbsystems.com/fullchain.pem
-rwxr-x--- 1 root root 1765 Nov 15 18:34 ispserver.csr
lrwxrwxrwx 1 root root 53 Nov 15 18:45 ispserver.key -> /etc/letsencrypt/live/ns10.cdbsystems.com/privkey.pem
-rwxr-x--- 1 root root 3243 Nov 15 18:45 ispserver.key-20251115184509.bak
lrwxrwxrwx 1 root root 53 Oct 1 2020 ispserver.key.secure -> /etc/letsencrypt/live/ns10.cdbsystems.com/privkey.pem
-rwxr-x--- 1 root root 5288 Nov 15 18:45 ispserver.pem
-rwxr-x--- 1 root root 7188 Nov 15 18:45 ispserver.pem-20251115184509.bak
[root@ns10 ssl]#

do I merely need to copy the 2 .bak files onto ispserver.key and ispserver.pem?


and postfix ssls will be wrong too I'm sure! I'm TOO TIRED have been reloading websites from the recovered drive for hours and hours
testing my mail server has it report it has a self signed cert.
can I force certbot to renew ns10.cdbsystems.com in this case??
thanks till. i'm going to BED

 

I tried unpacking the backup made by your code at that link - restoring them did NOT fix anything. still thinks an ns11 certificate? what the heck??

 

Seems fine, what do you want to do? Do you still want to delete the certs? If so, revoke them first, then delete and retry ISPConfig force update and create LE ssl certs thereafter. This is how you revoke using certbot: https://letsencrypt.org/docs/revoking/#from-the-account-that-issued-the-certificate

 

my cert is screwed - ns10.cdbsystems.com and ns10.cdbsystems.com:8080 both are self signed. I should never have touched ns10 I'm just too dang tired.
the cert31.pem file is dated TODAY and its the self-signed cert. I copied the *30.pem files to the *31.pem files.
now it says cert needs to be renewed. (as it thinks its aug3 I think).
certbot does not exist. certbot-auto is in /opt/certbot.
can I force renewal with
./certbot-auto renew --cert-name ns10.cdbsystems.com
--dry-run says it works...
will this save me? I am JUST so tired....

 

ok... ./certbot-auto command did renew ns10 and all seems to be working about! now about ns11.... geez I'm TIRED!

 

ok back to ns11. apache2 loads, but ns11.cdbsystems.com is not present as a vhost so we cant get to it.
there is no ns11.cdbsystems.com vhost in sites-available (or enabled).

can I create a vhost using ns10.cdbsystems.com as a template?
ns11 is the hostname on this server.
can I recreate these files intelligently? I'm tired of floundering around!
and if not can I 'delete' ispconfig and reinstall it without losing the varioius websites i've already migrated?
till help me

 

There is a very important warning missing in that tutorial!
You should NEVER EVER create a website with your server's hostname!
It will break renewal of your system's certificate in /usr/local/ispconfig/interface/ssl with all kinds of consequences once it expired!

If one wants extra hostnames in their system's certificate (for services that use that certificate) just do this:
https://forum.howtoforge.com/threads/lets-encrypt-procedure-questions.94718/#post-468472

 

The panel will be fine, as the webserver will be reloaded on renewal.
And postfix and dovecot will be fine too, having followed the tutorial.

But a service like pure-ftpd for example will fail you eventually when connecting with ftps if you don't take that service into account when following the tutorial, since /usr/local/ispconfig/server/scripts/letsencrypt_renew_hook.sh isn't run anymore on renewal.
The tutorial only focesses on mail services and doesn't warn you about other services that may be using the system certificate.

 

A far better solution would be to let run /usr/local/ispconfig/server/scripts/letsencrypt_renew_hook.sh in the tutorial's automatic renewal script creation instead of restarting postfix and dovecot services and do the symlink of the certificate and key the other way around.
(first copying it from /var/www/mail.example.com/ssl to /usr/local/ispconfig/interface/ssl)
That way it wouldn't change anything in ispc's default behaviour when it comes to the system certificate.

A far easier solution would be to not create a website and scripts but just add the extra hostnames one wants as alt hostnames in the certificate's config file and do a force renewal, to keep ispc's default behaviour of system certificate's renewal intact.

The tutorial is perfectly fine for postfix and dovecot certificate replacement when it DOESN'T contain the system's hostname.
(Skipping the symlink to /usr/local/ispconfig/interface/ssl !!)