Letsencrypt failure on migraiton between servers

Discussion in 'ISPConfig 3 Priority Support' started by tlove, Jun 15, 2025.

  1. tlove

    tlove Member HowtoForge Supporter

    Migration is a server update on same public IP. Migraiton is Ubuntu 22.04/ISPCONFIG 3.2 to Ubuntu 24.04/ISPCONFIG 3.3.
    Problem: all domains have no letsencrypt certificates on transfer and this is not repaired by manually renewing SSL and Lesencrypt in the console
    ispconfig.log shows 'Could not verify domain XXXX, so excluding it from letsencrypt request'
    There is no .well-known/acme-challenge/ directories in web directories of websites.
    There are no letsencrypt logs in /etc/letsencrypt or in /root/acme.sh
    Migration gave warning that target and source have different letsencrypt so could not transfer certificates.
    I welcome advice.
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    That's ok as no such directory is needed in the website.

    That's the reason for your problem. As the migration tutorial states, as a prerequisite for a migration, if you want the LE certs to be migrated, then the old and new servers must use the same LE client. You are using a different LE client between the old and new server, so LE certs could not be transferred.

    To fix this, you must uncheck Let's Encrypt checkbox in the website (if izs active), save, and enable it again and save.

    Please follow the Let's Encrypt FAQ step-by-step to find out why your new server is not able to create new LE certs:

    https://forum.howtoforge.com/threads/lets-encrypt-error-faq.74179/

    In your case, especially check that the domains point to the new server already, you will not get any certs when the domains still point to the old system, and ensure Migration mode is off (step 10 of the checklist).
     
  3. tlove

    tlove Member HowtoForge Supporter

    When I follow the above , when the certificate is being made during ispconfig_update.sh --force I get the message
    'Server's public ip(s) (xxx.xxx.xxx.xxx) not found in A/AAAA records for xxx.yyy.org: 127.0.1.1'
    However, nslookup confirms the public ip address of the server is correct and checking Route53 shows the same.
    Any ideas what I should check to resolve this?
     
  4. tlove

    tlove Member HowtoForge Supporter

    To give a bit more background. Both servers are connected in parallel to the same unifi firewall in the same webserver zone. They are both on 192.168.6.xxx and I simply flip the portforwarding between them. In Route53, both servers are subdomains of the same domain. The old server works perfectly (except it is almost out of hard drive). All else is the same except the new one doesn't make certificates. The hosting zones for the new server were copied directly from the old server and can be directly compared item by item and look to be identical. I've checked multiple times now over the last two and a half days. There must be a difference somewhere but I haven't been able to find it. I welcome suggestions.
     
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Yes, please start to follow the checklist from the beginning to the end:

    https://forum.howtoforge.com/threads/lets-encrypt-error-faq.74179/

    Do not stop after the first few points; read and test all points, especially after posting the debug output.

    Also, using ispconfig_update does not help with website SSL issues, it's about the ISPConfig central SSL cert for mail and FTP, and the control panel port only. Receiving 127.0.0.1 for the hostname is ok if you have this in your hosts file; the updater said already that you can accept that.
     
    Last edited: Jun 15, 2025
  6. tlove

    tlove Member HowtoForge Supporter

    The original email server is down. Email clients are unable to get certificates after upgrading ISPConfig 3.2 to 3.3. The priority is to get the original ISPConfig server functioning again. Can retry the migration later. I'm happy if original server goes back to normal (i.e. no more mail., smtp. and imap. subdomains for the email server) . Currently, the webserver is working ok and can log into the console with self-certificate.
    Have followed above debug checklist completely as attached below.
    1. Both letsencrypt and certbot installed – seen in directories and output from update.sh
    2. System is updates and ugpraded
    3. ISPCONFIG upgrade 3.2->3.3 resulted in failure of certificates for email and server (Email server is down) and problems accessing console.
    4. Skip Letsencrypt check is enabled
    5. No Cloudflare
    6. All domain names checked on Route53 and by dig and nslookup -all ok and their dns links them to server public IP. The host and domain and mail./smtp./imap.domain websites that were previously mistakenly created in ISPCONFIG console have been removed.
    7. All sites in console have * rather than IP address
    8. Server is using apache2.4
    9. Ispconfig_update.sh –force resulted in the following message:
      "Discovered certbot version 1.21.0 with certificate home /etc/letsencrypt
      Discovered acme.sh version 3.1.2 with certificate home /root/.acme.sh
      Using certificate path /etc/letsencrypt/live/khidr.alkhayyam.org / /etc/letsencrypt/live/khidr.alkhayyam.org/cert.pem
      ISPConfig currently is using a self-signed certificate.
      Using apache for certificate validation
      Issuing certificate via acme.sh failed. Please check that your hostname can be verified by letsencrypt
      Could not issue letsencrypt certificate, falling back to self-signed."
    10. Server migration mode is not ticked
    11. Port 80 is ALLOW in firewall of ISPCONFIG, in ufw and in unifi firewall. Also all websites are working fine
    12. tail -f /var/log/letsencrypt/letsencrypt.log results in
      File "/snap/certbot/4737/lib/python3.12/site-packages/certbot/_internal/main.py", line 1879, in main
      return config.func(config, plugins)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/snap/certbot/4737/lib/python3.12/site-packages/certbot/_internal/main.py", line 1627, in renew
      renewed_domains, failed_domains = renewal.handle_renewal_request(config)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/snap/certbot/4737/lib/python3.12/site-packages/certbot/_internal/renewal.py", line 667, in handle_renewal_request
      raise errors.Error(
      certbot.errors.Error: 1 renew failure(s), 3 parse failure(s)
      2025-06-16 12:58:12,895:ERROR:certbot._internal.log:1 renew failure(s), 3 parse failure(s)Enabled Debug.
    13. tail -f /var/log/ispconfig/acme.log results in:
      [Mon Jun 16 05:52:45 PM AWST 2025] GET
      [Mon Jun 16 05:52:45 PM AWST 2025] url='https://api.github.com/repos/acmesh-official/acme.sh/git/refs/heads/master'
      [Mon Jun 16 05:52:45 PM AWST 2025] timeout=30
      [Mon Jun 16 05:52:45 PM AWST 2025] _CURL='curl --silent --dump-header /root/.acme.sh/http.header -L -g --connect-timeout 30'
      [Mon Jun 16 05:52:45 PM AWST 2025] ret='0'
      [Mon Jun 16 05:52:45 PM AWST 2025] Already up to date!
      [Mon Jun 16 05:52:45 PM AWST 2025] Upgrade successful!
      [Mon Jun 16 05:52:45 PM AWST 2025] LE_WORKING_DIR='/root/.acme.sh'
      [Mon Jun 16 05:52:45 PM AWST 2025] Running cmd: setdefaultca
      [Mon Jun 16 05:52:45 PM AWST 2025] Changed default CA to: https://acme-v02.api.letsencrypt.org/directory

    14. There is no /root/.acme.sh/acme.sh.log
      Directory /root/.acme.sh/ has list of directories of the websites containing certs with almost all dates old in 2022 and 2023
    15. Enabled debug and commented out crontab as required
    16. Manually ran /usr/local/ispconfig/server/server.sh. Output was
      16.06.2025-10:16 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'.
      16.06.2025-10:16 - DEBUG [server:224] - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
      finished server.php.
      I welcome advice as I've no idea what to do next.
     
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    You must enable the Let's Encrypt checkbox on the website before running the server.sh.
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    Also, it seems as if you might have manually installed a second LE client (e.g. you have a system that uses acme.sh and you then manually installed certbot). You can not have certbot and acme.sh installed at the same time. Or you manually copied over the/etc/letsencrypt folder on a system with acme.sh, which has a similar effect that the system is not able to renew certificates.
     
  9. tlove

    tlove Member HowtoForge Supporter

    Thank you for your messages. The websites are all running fine with certificates. The problem is the certificate for server itself and the subsequent certificates for emails.The email clients are all rejecting all the email certificates for Imap and smtp so no one is able to use email. Its also not clear now what server url people should be entering as the name of the server in their email config. Is domain.com or host.domain.com. Currently, neither work. Or should it be the name of the server that connects with their email? Previously everyone used the domain of the server.
    You are correct about the doubling up on certbot and acme. This happened at a migration 4-5 years ago. I seem to remember that the early system was using certbot and the new system used acme. How is best to sort this out?
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    By default, the SSL certificate for the mail services is issued for the hostname of the server. This means you must use the server hostname as the SMTP and IMAP server in your email client. If this differs from your previous setup, then you must have changed that manually in your previous setup, e.g., to use the email certificate of a certain website for email instead of the ISPConfig certificate. In that case, you must do the same manual configuration on your new system that you did on the old system to achieve the same system behaviour.

    In the title, you said the LE issues are from a migration of servers (migration means you transferred an ISPConfig setup to new hardware e.g. using ISPConfig Migration Tool), in the last post, you say they are from an ISPConfig update from 3.2 to 3.2. Did you do a server migration, or did you do an ISPConfig update?
     
  11. tlove

    tlove Member HowtoForge Supporter

    Around 2000 was a migration between hardware (server A to server B). That was when the Acme/Certbot mess arrived. Same year I reconfigureed the email domains and certificates following a HowToForge tutorial to create mail.domain.com, smtp.domain.com, Imap.domain.com etc. This worked except that the console didn't have a certificate.
    Around 2023 you gave me some instructions for symlinks so that server B would have a certificate and it would renew. That worked well.
    This week I tried to do a migration from Server B to a new Server C. That migration failed. One suggestion I found a couplf of days ago was to first upgrade the source server (server B) from ISPConfig 3.2 ->3.3. After the upgrade on Server B the email certificates failed so no one has been able to use email. I've spent 2 days unsuccessfully trying to work out what is wrong and what would fix it. AI hasn't helped.
    I'm first trying to get server B working again before doing the migration to server C.
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    Okay, this makes things clearer. First, an Update of ISPConfig on the old system is never required for a migration, and I highly recommend against doing such an update.

    You seem to use a website for SSL certificates for the email system. When you access this website in a browser using the subdomains you used for email, does it still work with a valid SSL certificate?
     
  13. tlove

    tlove Member HowtoForge Supporter

    None of the email websites have a certificate and all of them unexpectedly open an expired website that no longer has a domain registered for it.

    If it makes it simpler, I would very much prefer to go back to whatever is the standard configuraiton for ISPConfig for the emails.
     
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    I wonder if it might be best to contact Thom from ISPConfig business support here: https://www.ispconfig.org/get-support/?type=ispconfig to clean this up. The mix of different LE clients plus manually altered setup plus there seems to be issues in certbot itself reading his config files make things quite complex and I guess the best solution would be that someone cleans this up on your server directly. I had a quick chat with Thom and he is available at the moment to take a look on your system to fix this.
     
  15. tlove

    tlove Member HowtoForge Supporter

    Thanks Till sounds a good idea. I'll contact him straight away.
     
  16. tlove

    tlove Member HowtoForge Supporter

    Hi Till, I used the support link to send a message to Thom. If you see him please could you pass a message that I'm in Western Australia and its very late at night here.
     
  17. till

    till Super Moderator Staff Member ISPConfig Developer

    You can also send him a PM here if you want @Th0m I'll send him a message.
     

Share This Page