Securing ISPConfig 3 Control Panel (Port 8080) With Let's Encrypt Free SSL

Discussion in 'Tips/Tricks/Mods' started by ahrasis, Feb 14, 2017.

  1. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Securing ISPConfig 3 Control Panel (Port 8080) With Let's Encrypt Free SSL
    [Introduction]

    I rewrote this since my LE4ISPC script has matured and now automatically create Let's Encrypt for any ISPConfig server hostname FQDN if none exists; and secure its control panel and other services; if they are available and installed; so some of the original post will be removed or modified.

    To avoid confusion, do not use this guide if you manually follow the old tutorial at its page or as attached below this post, as this guide uses LE4ISPC script approach.

    Also note that we do all these in sudo mode using nginx as example but it will also work for apache2 and other ISPConfig server.

    [Creating ISPConfig Server Website Using Its Hostname FQDN]
    [Accessing ISPConfig Website Online]

    We don't have to do the above anymore as we no longer use certbot webroot.

    [Securing ISPConfig Server hostname FQDN With Let's Encrypt SSL]
    We now use certbot standalone instead.
    Code:
    lelive=/etc/letsencrypt/live/$(hostname -f)
    if [ ! -d "$lelive" ]; then
       certbot certonly --authenticator standalone -d $(hostname -f) --pre-hook "service nginx stop" --post-hook "service nginx start"
    fi
    
    Replace nginx with apache2 if you use the later. Furthermore, if this is not a web server, remove from --pre-hook onwards from the above command.

    [Assigning Let's Encrypt SSL For ISPConfig 3.1 Control Panel, If You Have It]
    When LE SSL certs are issued, symlink ISPConfig SSL files with Let's Encrypt SSL files.
    Code:
    ispccrt=/usr/local/ispconfig/interface/ssl/ispserver.crt
    ispckey=/usr/local/ispconfig/interface/ssl/ispserver.key
    ispcpem=/usr/local/ispconfig/interface/ssl/ispserver.pem
    ln -sf $lelive/fullchain.pem $ispccrt
    ln -sf $lelive/privkey.pem $ispckey
    cat $ispckey $ispccrt > $ispcpem; chmod 600 $ispcpem
    

    [Enabling SSL For ISPConfig 3.1 Control Panel]
    We now use a simpler way via sed command on ispconfig.vhost file e.g. for nginx:
    Code:
    sed -i "s/ssl off/ssl on/g" /etc/nginx/sites-available/ispconfig.vhost
    sed -i "s/#ssl_/ssl_/g" /etc/nginx/sites-available/ispconfig.vhost
    
    E.g. for apache2:
    Code:
    sed -i "s/#SSL/SSL/g" /etc/apache2/sites-available/ispconfig.vhost
    sed -i "s/SSLCACertificateFile/#SSLCACertificateFile/g" /etc/apache2/sites-available/ispconfig.vhost

    [Checking LE SSL For ISPConfig 3.1 Control Panel]
    Just go to your ISPConfig 3.1 Control Panel page e.g. https://server.domain.tld:8080 to verify.

    [Using The Same Let's Encrypt SSL Certs For Other Major Services]
    Based on Securing Your ISPConfig 3 Installation, we use symlink to ispserver.key or .crt or .pem instead of directly pointing your postfix, dovecot, courier, pure-FTPd and monit to the same SSL files.
    a. For postfix:
    Code:
    cd /etc/postfix/
    ln -sf /usr/local/ispconfig/interface/ssl/ispserver.crt smtpd.cert
    ln -sf /usr/local/ispconfig/interface/ssl/ispserver.key smtpd.key
    service postfix restart; service dovecot restart
    
    b. For dovecot: (* Note this shouldn't exist together with courier)
    Check if it is set to use the postfix SSL files (see below) via "nano /etc/dovecot/dovecot.conf".
    Code:
    [...]
    ssl_cert = </etc/postfix/smtpd.cert
    ssl_key = </etc/postfix/smtpd.key
    [...]
    Leave it, if it is rightly set. Otherwise, fix it. In any event, running "service dovecot restart" is already covered earlier.
    c. For courier: (* Note this shouldn't exist together with dovecot)
    Code:
    cd /etc/courier/
    ln -sf /usr/local/ispconfig/interface/ssl/ispserver.pem imapd.pem
    ln -sf /usr/local/ispconfig/interface/ssl/ispserver.pem pop3d.pem
    service courier-imap-ssl stop; service courier-imap-ssl start
    service courier-pop-ssl stop; service courier-pop-ssl start
    
    d. For MySQL / MariaDB: (#247)
    Code:
    mcrt=/etc/mysql/server-cert.pem
    mkey=/etc/mysql/server-key.pem
    mcnf=/etc/mysql/my.cnf
    scp /usr/local/ispconfig/interface/ssl/ispserver.crt $mcrt
    scp /usr/local/ispconfig/interface/ssl/ispserver.key $mkey
    chown mysql:mysql /etc/mysql/server*.pem
    chmod 400 /etc/mysql/server*.pem
    sed -i "/\[mysqld\]/a ssl-cipher=TLSv1.2\nssl-cert=$mcrt\nssl-key=$mkey" $mcnf
    service mysql restart
    
    e. For pure-FTPd:
    Code:
    cd /etc/ssl/private/
    ln -sf /usr/local/ispconfig/interface/ssl/ispserver.pem pure-ftpd.pem
    chmod 600 pure-ftpd.pem; service pure-ftpd-mysql restart
    f. For monit: (If it is installed in your server)
    Code:
    monitrc=/etc/monit/monitrc
    ftpdpem=/etc/ssl/private/pure-ftpd.pem
    sed -i '/PEMFILE/d' $monitrc
    sed -i "s@SSL ENABLE@SSL ENABLE\n\tPEMFILE $ftpdpem@" $monitrc
    service monit restart
    

    [Creating Auto Renewal Script For Your ISPConfig Pem File (ispserver.pem)]
    In this last step, use incron as suggested in the incron tutorial and create a script to automatically update our ispserver.pem file.
    a. Via terminal command, install incron, then create the script file and edit it using nano:
    Code:
    apt install -yqq incron
    nano /etc/init.d/le4ispc_pem.sh
    b. Add this in the le4ispc_pem.sh:
    Code:
    #!/bin/sh
    cd /usr/local/ispconfig/interface/ssl/
    mv ispserver.pem ispserver.pem-$(date +"%y%m%d%H%M%S").bak
    cat ispserver.{key,crt} > ispserver.pem
    chmod 600 ispserver.pem; chmod 600 /etc/ssl/private/pure-ftpd.pem
    service pure-ftpd-mysql restart; service monit restart
    service postfix restart; service dovecot restart
    service mysql restart; service nginx restart
    c. We then make it executable, add root as allowed user for incrontab and then edit incrontab file:
    Code:
    chmod +x /etc/init.d/le_ispc_pem.sh
    echo "root" >> /etc/incron.allow
    echo "/etc/letsencrypt/archive/$(hostname -f)/ IN_CREATE,IN_MODIFY /bin/bash /etc/init.d/le_ispc_pem.sh" >> /var/spool/incron/root
    

    [Restarting Your System]
    Restart your webserver system afterwards, it this is a web server. If you are using apache, change nginx to it accordingly.
    Code:
    service nginx restart

    [LE4ISPC SCRIPT]
    We have cover all this and more in our LE4ISPC script created specifically for ISPConfig servers, so to ease many of us, please use the script instead.

    In your terminal, in root mode, simply run this after you have installed any of your ISPConfig server build:
    Code:
    cd /etc/ssl
    wget https://raw.githubusercontent.com/ahrasis/LE4ISPC/master/le4ispc.sh --no-check-certificate
    chmod +x le4ispc.sh
    ./le4ispc.sh

    [Special note]
    Thank you @till, @florian030, @Thaddeus, @Taleman, @Turbanator, @Jesse Norell, @ztk.me and others for their observations, feedbacks and suggestions that lead to the improvement of LE4ISPC script.
     

    Attached Files:

    Last edited: Dec 14, 2023
  2. Turbanator

    Turbanator Member HowtoForge Supporter

    This is great, thank you. I think your cron line has a typo. le_ispcon_pem.sh should be le_isp_pem.sh
     
    MaKa likes this.
  3. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Yes. You are right. Thank you for the fix. I already updated the guide with your fix. Thank you again.
     
  4. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    Clever, I like it.

    Doesn't this end up with multiple inotifywait processes watching the privkey1.pem file, with a new one starting every day? (Maybe it prints a notice that it's running and exits, I've not tried it yet.) Might need le_ispc_pem.sh to use a lock file or something?

    You also need to restart nginx, pure-ftpd and monit whenever the certificate changes, maybe make it run as part of the loop in le_ispc_pem.sh.
     
  5. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Though I have also said the above, there is always room for improvement. Any suggestion for that?

    By the way, I am not so sure why we need to restart nginx, pure-ftpd and monit whenever the certificate changes. To me they are just using symlink to ISPC ispserver.pem and that symlink will remains after any changes. Care to explain why since the symlink name remains the same even after the renewal?
     
  6. till

    till Super Moderator Staff Member ISPConfig Developer

    The services won't recognize and pick up a changed SSL cert without restart. An real world example: we use LE as cert authority on the git.sipconfig.org server, the server runs nginx and gitlab without a controlpanel, during initial installation we missed to add a restart of nginx after cert renewal which caused an expired SSL cert warning in the browser while the actual cert was up to date. Adding a nginx restart fixed that.
     
    Fire Fox and ahrasis like this.
  7. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Thank you till and Jesse Norell for their observations, feedbacks and suggestions that lead to the improvement of this guide.

    I changed the script to be an init.d script, add new header to it as well as -m to inotifywait and and service nginx restart as thing to be done upon such changes. The reason for its change to an init.d script is due to the fact that inotifywait may stop on system reboot unless it is made a part of update-rc.d. With this change there will be no need of the tedious cron job.

    I am still not sure if we should restart pure-ftpd and monit services as well. Please do explain why it is needed, if you want it to be added as well.

    Thank you again.
     
    Fire Fox likes this.
  8. florian030

    florian030 Well-Known Member HowtoForge Supporter

    Maybe we can add pre and post-hooks to the certbot-call. A post-hook could restart several services like mail and ftp. Just LE decided, if there is a need for restarting a services.
     
    ahrasis and Fire Fox like this.
  9. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Interesting idea florian. I'd prefer a hook to LE as well but that would require a proper knowledge and experience with its code of which I do not have. I will leave that to the expert hands to build one.
     
    Fire Fox likes this.
  10. florian030

    florian030 Well-Known Member HowtoForge Supporter

    If you want to try post-hooks:
    Download 900-letsencrypt.inc.php.gz and unzip the file to /usr/local/ispconfig/server/lib/classes/cron.d
    Create a post-hook-file /usr/local/ispconfig/server/scripts/le_post-hook.sh. I.e.
    Code:
    #!/bin/sh
    service postfix restart
    service dovecot restart
    and set the permissions:
    Code:
    chown root.root /usr/local/ispconfig/server/scripts/le_post-hook.sh
    chmod 700 /usr/local/ispconfig/server/scripts/le_post-hook.sh
    
    The post-hook will be called every time a cert is renewed. Maybe it's possible to limit the restarts to several domains.
     
    girthh likes this.
  11. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Great code. So, basically, we need to modify the default 900-letsencrypt.inc.php or replace it with the one you provide. This is creating one question - will it be replaced by an ISPConfig update?

    And if the postfix and dovecot restart is necessary, (because for now I still do not think they are yet, similar to pure-ftpd and monit), it would rather be easier if they are included in the suggested script in the above guide.
    Code:
    while inotifywait -m -q -e modify,attrib /etc/letsencrypt/archive/yourserverdomain/privkey1.pem >/dev/null; do
      rm /usr/local/ispconfig/interface/ssl/ispserver.pem
      cat /usr/local/ispconfig/interface/ssl/ispserver.{key,crt} > /usr/local/ispconfig/interface/ssl/ispserver.pem
      service nginx restart
      [ADD ANY EXTRA / NEEDED CODE IN HERE]
      [ADD ANY EXTRA / NEEDED CODE IN HERE]
      [ADD ANY EXTRA / NEEDED CODE IN HERE]
    done[code]
     
  12. florian030

    florian030 Well-Known Member HowtoForge Supporter

    An ispconfig-update will overwrite this. It's more like a quick hack but if this is working, i will add this to the main-code.
    With a post-hook you can trigger an action if you you get new cert. Your script checks only a specified cert. I think, it's easier to just define some hooks in a script. Especially, when you run a multiserer-setup with just one master and one interface. It should possible to copy new cerst over ssh with a hook-script to another server.
    You have to reload postfix and dovecot after you changed the cert. You may want to add the bundle to postfix and dovecot, too. I saw some problems with outlook when the bundle was not defined.
     
  13. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Great. It would be nicer if by default, ISPConfig could create its own Let's Encrypt SSL, renew it and then restart all necessary services upon its Let's Encrypt SSL files renewal.

    Note that I did not include restart the system earlier because I was under the impression that ISPConfig should restart it after any Let's Encrypt SSL renewal. Then, after reading the replies, I realized that ispserver.pem may not be deleted and rebuilt at that time yet, so I added it in, just to make sure.

    However, I am still in doubt to add restart for other services though adding them will be more prudent. So in the above guide, I added restarting them as an option instead.

    By the way, what do you mean by bundle? If you mean the certificate and the chain, that are already included, because from what I understand, they are already bundled in the fullchain.pem. So in the above guide, I used:
    Code:
    ln -s /etc/letsencrypt/live/yourserverdomain/fullchain.pem /usr/local/ispconfig/interface/ssl/ispserver.crt
     
    Last edited: Feb 16, 2017
  14. till

    till Super Moderator Staff Member ISPConfig Developer

    I doubt that any service will recognize a changed SSL cert without restart as these certs were normally read only once when the service starts and it is kept in memory then, so it is very likely that you have to restart each service that uses this SSL cert on renewal.
     
    ahrasis likes this.
  15. till

    till Super Moderator Staff Member ISPConfig Developer

    Bundle is another term for chain certificate.
     
    ahrasis likes this.
  16. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    Thank you for the feedback. I edited the guide to for all them to be included and restarted upon Let's Encrypt SSL renewal (not as option anymore).

    So, I get that bundle as already included and correct. Thank you till.
     
    Last edited: Feb 16, 2017
  17. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    As @till has indicated, anything that holds the certificate in memory probably needs restarted; pure-ftpd does for sure, I have no experience with monit.

    On the design of implementation from ispconfig triggers, I would probably not use inotifywait myself. Normally letsencrypt will renew the certificate with there is still a fair bit of time (I think 1 month?) until the old certificate expires - you definitely don't need up to the second service reloads after the certificate changes. As a standalone monitoring script I think it is clever and would have very quick response to certificate changes. For an ispconfig hook like @florian030 is suggesting I would compare the pure-ftpd pid file timestamp with the certificate timestamp, and only restart pure-ftpd if the certificate is newer.
     
  18. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    For me, I will not use the suggested replacement of 900-letsencrypt.inc.php until it is implemented by ISPConfig, just in case I need to update ISPConfig as it might wipe out the replacement.

    And on the contrary, I do believe inotifywait will check and only acted upon a change of the specified file and this includes the attributes. So it is basically safer and more practical for the time being.
     
  19. Tuumke

    Tuumke Active Member

    I use ISPConfig for personal websites only at the moment.
    How would use my mail setup so that it sees the letsencrypt certs?
    Currently each domain has its own A 3600 mail.domain.tld pointing to the server ip.
    The MX record points to the A 3600 mail.domain.tld

    Do i have to setup the auto-subdomain to * or something so that the ssl cert is valid for domain.tld and *.domain.tld?
     
  20. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    ISPConfig currently only requests letsencrypt certificates for websites, so use with email must be setup manually for now. The instructions in this thread only request a certificate for the server's full hostname (output of `hostname -f`); you can request a certificate which includes multiple names (server hostname, mail.domain1.com, mail.domain2.com, etc.) from the command prompt, which would work if you have a single server installation (not multi-server). You would need to request a new certificate from the command line every time you add or remove a name from it; as long as the first name in the certificate is `hostname -f`, then the instructions above to configure postfix and dovecot should work (ie. point to the certificate/key files in /etc/letsencrypt/live/`hostname -f`/).

    Note you can only have I believe 100 names included in a letsencrypt certificate, so doing this is only practical for small sites/config. Note also that if any of the domains you have move away from your server, so that mail.thatdomain.com no longer resolves to your server's ip address, then you must request another certificate without that name included; if you do not, the certificate cannot be renewed as it is, and will expire in < 90 days.

    Another option is to use a different letsencrypt client, acme.sh, and this would work in a multi-server installation. It's a little more complicated to setup, but not too bad; @sjau has posted a howto and various other info on it on the forums here. It sounds like IPSConfig may switch to using acme.sh in the future (or maybe support both clients).

    If interested, our current solution to this is to tell customer they can use mail.theirdomain.com as the server name for convenience, but they will get certificate errors; or they can use the actual server hostname without errors, but will probably have to change that every now and then (eg. every few years) whenever their domain moves to a new server. Some pick/prefer one, some the other, and I think most don't care.
     
    Gwyneth Llewelyn likes this.

Share This Page