Emergency: acme.sh and its directory vanished

Discussion in 'ISPConfig 3 Priority Support' started by hotmifi, May 20, 2026 at 4:27 PM.

Tags:
  1. hotmifi

    hotmifi Member HowtoForge Supporter

    Im runnig a production system on Debian Trixie. Yesterday an SSL certificate was up for renewal. But it did not renew, acme.sh is wiped from the disk instead.
    The only mentioning of acme.sh I could find is in crontab:
    "41 18 * * * "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh" > /dev/null"
    Certificates are in place, but expired, so I cannot even start the ISPconfig home page.
    Is there any way to recover or do I have to reinstall ISPconfig from scratch and restore the production website?
    Regards;
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    Are you sure it's not there? It's a hidden folder, you will not see it with plain ls command. Check with:

    ls -la /root/.acme.sh/

    also, if your system is older, it might use certbot and not acme.sh
     
  3. hotmifi

    hotmifi Member HowtoForge Supporter

    cat /etc/os-release
    PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
    NAME="Debian GNU/Linux"
    VERSION_ID="13"
    VERSION="13 (trixie)"
    VERSION_CODENAME=trixie
    DEBIAN_VERSION_FULL=13.5
    ID=debian
    HOME_URL="https://www.debian.org/"
    SUPPORT_URL="https://www.debian.org/support"
    BUG_REPORT_URL="https://bugs.debian.org


    The whole configuration is not older than 3 months. And I know it's hidden or at least was.
    Here are the remains:

    ls -alht
    total 56K
    drwx------ 7 root root 4.0K May 20 16:38 .
    -rw------- 1 root root 10K May 20 16:30 .bash_history
    drwxr-xr-x 18 root root 4.0K May 20 16:21 ..
    -rw------- 1 root root 49 May 20 16:17 .lesshst
    -rw-r--r-- 1 root root 66 Feb 24 19:58 .selected_editor
    -rw-r--r-- 1 root root 3.5K Feb 24 19:14 .bashrc
    -rw-r--r-- 1 root root 807 Feb 24 19:13 .profile
    drwx------ 2 root root 4.0K Feb 24 18:59 .ssh
    drwx------ 4 root root 4.0K Feb 24 12:50 .config
    drwx------ 4 root root 4.0K Feb 24 12:50 .cache
    drwxr-xr-x 3 root root 4.0K Feb 23 19:02 .local
    drwxr-xr-x 4 root root 4.0K Feb 23 17:01 oriweb
    and

    ll /var/www/opse.wbiat.de/ssl/
    total 8
    -rw-r--r-- 1 root root 2848 Feb 19 18:18 opse.wbiat.de-le.crt
    -rw------- 1 root root 227 Feb 19 18:18 opse.wbiat.de-le.key (Feb 19 was the installation day of ISPconfig, too.)


    ll /etc/postfix/smtpd.*
    lrwxrwxrwx 1 root root 48 Feb 19 13:46 /etc/postfix/smtpd.cert -> /usr/local/ispconfig/interface/ssl/ispserver.crt
    lrwxrwxrwx 1 root root 48 Feb 19 13:46 /etc/postfix/smtpd.key -> /usr/local/ispconfig/interface/ssl/ispserver.key

    I did not even touch the system yesterday, today the certificates expired and browsers refuse to access the website-
     
    Last edited: May 20, 2026 at 4:50 PM
  4. till

    till Super Moderator Staff Member ISPConfig Developer

    Do you have a server backup you can recover the folder from? If not, you can either try installing it manually, or run an ISPConfig update with:

    ispconfig_update.sh --force

    It should install acme.sh if its not there when you let it create a new ssl cert for ISPConfig.
     
  5. hotmifi

    hotmifi Member HowtoForge Supporter

    This is my first server of 40 or more, where I did not backup .acme.sh :mad:
    Okay, I'll try your advice. Still I'd like to know what really happened.
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    That's the scarier part, as only root user can remove it, and ISPConfig has no function to remove, update or reinstall acme.sh in the server side code. Only the updater can do that but the installer or updater can install acme.sh, but its code is not permanently on the system and it also can't remove it. Also, I have not seen a system yet where acme.sh vanished. But in the light of the verious recent Linux Kernel exploits, its a bit scary. But on the other hand, a hacker that just deletes acme.sh .... makes no sense to me.
     
  7. hotmifi

    hotmifi Member HowtoForge Supporter

    Running again. Thanks alot Hacker is nearly impossibe and highly unlikely. The only manually started root activity which has happened since installation was the upgrade to Trixie 13.5. How this could wipe the whole .acme.sh directory, no idea.
     
  8. hotmifi

    hotmifi Member HowtoForge Supporter

    Is some explanation in here?
    ll -th /var/log/ispconfig/acme*
    -rw-r----- 1 root root 65K May 20 18:41 /var/log/ispconfig/acme.log
    -rw------- 1 root root 29 May 20 00:02 /var/log/ispconfig/acme.log.1.gz
    -rw------- 1 root root 29 May 19 00:02 /var/log/ispconfig/acme.log.2.gz
    -rw------- 1 root root 29 May 18 00:02 /var/log/ispconfig/acme.log.3.gz
    -rw------- 1 root root 29 May 17 00:02 /var/log/ispconfig/acme.log.4.gz
    -rw------- 1 root root 29 May 16 00:02 /var/log/ispconfig/acme.log.5.gz
    -rw------- 1 root root 29 May 15 00:02 /var/log/ispconfig/acme.log.6.gz
    -rw------- 1 root root 29 May 14 00:02 /var/log/ispconfig/acme.log.7.gz
    -rw------- 1 root root 29 May 13 00:02 /var/log/ispconfig/acme.log.8.gz
    -rw------- 1 root root 29 May 12 00:01 /var/log/ispconfig/acme.log.9.gz
     
  9. till

    till Super Moderator Staff Member ISPConfig Developer

    That's the log of what acme.sh is doing; I'm quite sure you will not find any info there on why it vanished.
     
  10. hotmifi

    hotmifi Member HowtoForge Supporter

    These logs seem to be empty:

    -rw------- 1 root root 29 May 20 00:02 /var/log/ispconfig/acme.log.1.gz
    -rw------- 1 root root 29 May 19 00:02 /var/log/ispconfig/acme.log.2.gz
    -rw------- 1 root root 29 May 18 00:02 /var/log/ispconfig/acme.log.3.gz
    -rw------- 1 root root 29 May 17 00:02 /var/log/ispconfig/acme.log.4.gz
    -rw------- 1 root root 29 May 16 00:02 /var/log/ispconfig/acme.log.5.gz
    -rw------- 1 root root 29 May 15 00:02 /var/log/ispconfig/acme.log.6.gz
    -rw------- 1 root root 29 May 14 00:02 /var/log/ispconfig/acme.log.7.gz
    -rw------- 1 root root 29 May 13 00:02 /var/log/ispconfig/acme.log.8.gz
    -rw------- 1 root root 29 May 12 00:01 /var/log/ispconfig/acme.log.9.gz

    So the vanishing took place before May 12 and therefore not in connection with a renewal.
    How does ISPconfig handle acme.sh certificates not belonging to any defined site? Like if it looks a certificate has not been created initiated by ISPconfig?
     
  11. till

    till Super Moderator Staff Member ISPConfig Developer

    ISPConfig does not call acme.sh for a specific site/renewal. It just calls it once every night and acme.sh renews all certs that are due to be renewed on its own, no matter if they were certs from ISPConfig or not.
     
  12. hotmifi

    hotmifi Member HowtoForge Supporter

    Thank you very much. It will stay a mystery then and as I control the domain stealing certificates would be useless outside a LAN.
     

Share This Page