LetsEncrypt ECC Certificates with ISPconfig

Hi,
since ECC handshakes are faster than RSA, I wanted to provide them in the future. Have a few questions:
1) Did someone already manage to get them running with ispconfig and automated renewal?

As far as I have seen certbot does not support it, so I would have to switch to acme.sh. That seems to be supported by ispc, but I can't find any documentation (?).

2) Is it enough to remove certbot and install acme.sh (via official guide) or do I have to change some config / reconfigure services?
3) Where should I make changes (for example to choose cipher strength) to be compatible with future ispc changes?

Thanks, Johannes

Edit: Maybe I should bumb https://git.ispconfig.org/ispconfig/ispconfig3/issues/4315 ?

 

This a great news. I didn't realise this is already in 3.1 stable branch.

The code is in letsencrypt.inc.php. You can view it in the git too: https://git.ispconfig.org/ispconfig...le-3.1/server/lib/classes/letsencrypt.inc.php

 

Thanks for the reply. I have switched from certbot to acme.sh and will observe for a few days if everything still works as expected (I have some time, just renewed all certificates). Then I will try to add ECC certs.

Thanks! Am I right, that it should be enough to add "--keylength ec-256" to line 77 in the issue command? Then deactivate letsencrypt for a website and reenable it to issue a new cert? -> I will just try it with a new subdomain

I am not sure how the renew works. Is it correct that this is not done by ispconfig (directly), but by the acme.sh cron script?

 

Me too. Reading from the git, the letsencrypt cron job seems to cover certbot only, so, unless otherwise is confirmed, I do think acme.sh will run its own cron job. Check it out in here: https://git.ispconfig.org/ispconfig...er/lib/classes/cron.d/900-letsencrypt.inc.php

 

Small status update: Mail and Web is now running with ECC certificates. It is a bad hack, eventually I will post it here, but I think it will do more problems than gain because it does not survive an ispc update yet. Might help some developers(?).

Have two questions/remarks:

1. As far as I understand the sourcecode, the certs from acme.sh should be installed/copied to /var/www/clients/clientX/webXY/ssl (which is a good idea and a perfect place for them). However, mine are going to the standard directory (/root/.acme.sh/domain). Any idea why? (LE_WORKING_DIR is not set and standard is used).
   
2. There is no keylength set, so acme.sh requests 2048 bit RSA keys instead of 4096. Fixed for already issued certs with
   Code:

   for i in /root/.acme.sh/*.*/*.conf; do sed -i "s/Le_Keylength=''/Le_Keylength='4096'/g" "$i"; done

 

I think they are in the right place but they should also be symlinked to the site's ssl folder. However I am not sure of their exact location and structure as I haven't fully tested acme.sh with ISPConfig yet.

 

I have made a script to generate the symlink. However, in https://git.ispconfig.org/ispconfig...le-3.1/server/lib/classes/letsencrypt.inc.php function get_website_certificate_paths should get the paths and in request_certificates

Code:

324 $tmp = $app->letsencrypt->get_website_certificate_paths($data);
325        $domain = $tmp['domain'];
326        $key_file = $tmp['key'];
327        $crt_file = $tmp['crt'];

should get paths and give them to

Code:

402         if($use_acme) {
403            $letsencrypt_cmd = $this->get_acme_command($temp_domains, $key_file, $bundle_file, $crt_file, $server_type);

From acme.sh docu:

"this command" is the acme.sh --install, which is done in

Code:

77         $cmd = 'R=0 ; C=0 ; ' . $letsencrypt . ' --issue ' . $cmd . ' -w /usr/local/ispconfig/interface/acme ; R=$? ; if [[ $R -eq 0 || $R -eq 2 ]] ; then ' . $letsencrypt . ' --install-cert ' . $cmd . ' --key-file ' . escapeshellarg($key_file) . ' ' . $cert_arg . ' --reloadcmd ' . escapeshellarg($this->get_reload_command()) . '; C=$? ; fi ; if [[ $C -eq 0 ]] ; then exit $R ; else exit $C  ; fi';

But somehow this did not work. Just found out by doing this command on its own in the command line, that acme.sh does not replace symlinks. It follows the symlink and replaces that file. Since I had it symlinked to the acme.sh folder, it got a mess ,... So: first delete old symlinks, then use acme.sh --install ;-) Will try this later. However, after the manual command, this gets written to the config file:

Code:

Le_RealKeyPath='/var/www/mail.domain.de/mail.domain.de-le.key'
Le_RealFullChainPath='/var/www/mail.domain.de/mail.domain.de-le.crt'

So, something failed for the initialization. I think the problem was, that I have tried so much in the beginning, that issuing failed (50 certs/week limit). However, all acme.sh config was created and remained, just the "-- install" failed..

Note to mysqlf:

1. Make sure to not migrate all domains at once, to avoid letsencrypt limit
2. Delete old symlinks in /ssl/ before deploying acme.sh
3. Add keylength to issuing command

 

Hi, just want to give some feedback on what I did to get it working

Disclaimer:

• Apart from using ECC certificates, there is currently no reason to change from a working certbot installation
• In fact, you could run certbot for ispc managed websites and acme.sh for ecc certs in parallel
  
• This is not a guide, only follow if you know what you are doing
• Be aware: This will most likely break your websites for a few days

Changing from Certbot to acme.sh
In theory pretty simple: Remove certbot and all certificates. acme.sh will be automatically installed by ispc.
Challenge: Running system with working certificates should not have downtime. And that is the main challenge as you will see:
A problem is that letsencrypt has a limit of 50 requests per domain per week. If you have (like me) a lot of different subdomains it gets nasty. ISPC gets an own certificate for each subdomain and I want to have RSA and ECC certs in parallel, so 25 (sub)domains is the maximum per week. If you make one mistake or have updated certs in the last week...
If you only have a few domains, you could (probably) run certbot and acme.sh in parallel so that you switch when acme.sh has valid certs for all domains. But I have not tested this..
What I did:

• Preperation: Force update of all certificates via certbot that would expire soon
  
• Remove certbot (remove all of them, I had one installed via packages and one in /opt/eff, that was not even in $path)
• Update ispc (if you are at 3.1.15p2 or below)
  
• Wait one week (Hint: changes at websites could lead to problems now, better if you don't need to )
• Install acme.sh (I used official guide, but in principle ispc should install it when no certbot is found)

Now it is time to get all the certificates via acme.sh. There are several methods to do this, I have not found a good one :/
For a few websites you can do it manually. For all websites what I did was ISCP -> Tools -> Resync Web (which recreates all certificates via acms.sh). However, if that fails (and it did for me) then apache won't start, websites will disable SSL, etc ...
There are also a few more pitfalls:

• When acme.sh issue command fails, then the config for that domain is created, but the certificate is not installed in the web (and will not automatically in the future). ispc should maybe delete the config, if issuing fails
• During installation, acme.sh copies the certs into the webs ssl folder. However, I had a symlink to the old certbot certificates there. Strange things happened for me when acme.sh tried to install the cert there (the symlink was not replaced but the source)

Maybe the best (but slow) solution would be to first deactivate SSL for the web. Remove symlink in ssl folder. Reactivate SSL/LetsEncrypt which will issue and install a new certificate via acme.sh.
My Method for all webs:
Edit /usr/local/ispconfig/server/lib/classes/letsencrypt.inc.php in line 77 which currently reads:

Code:

$cmd = 'R=0 ; C=0 ; ' . $letsencrypt . ' --issue ' . $cmd . ' -w /usr/local/ispconfig/interface/acme ; R=$? ; if [[ $R -eq 0 || $R -eq 2 ]] ; then ' . $letsencrypt . ' --install-cert ' . $cmd . ' --key-file ' . escapeshellarg($key_file) . ' ' . $cert_arg . ' --reloadcmd ' . escapeshellarg($this->get_reload_command()) . '; C=$? ; fi ; if [[ $C -eq 0 ]] ; then exit $R ; else exit $C  ; fi';

If you want (I wanted), add "--keylength 4096" to the issue command to get 4096 bit instead of 2048 bit RSA keys. Copy the first command and this time use "--keylength ec-384"
Remove the second command (with --install-cert). (will be added again later)
Then do ISCP -> Tools -> Resync Web. All the config files will be written.
Reinsert the install-cert command again and copy this too, to install the ecc files in the right place
(Maybe would be better to copy the whole command to also install the ECC certs. Will edit this in the future)

Verify with "acme.sh --list" that all certificates are there and issued (date at "created"). If not, force renew ("acme.sh --renew --force -d domain", append --ecc for ECC cert ). If you get a rate limit error, wait a day (until week) and try again.
Backup the modified letsencrypt.inc.php file, it will be overwritten with future ispc versions


If all certificates are issued. Now lets install them.
I used this script to a) remove old symlinks b) install RSA certs c) install ECC certs: (It is quick'n dirty but worked for me)

Code:

#!/bin/bash
for d in /var/www/*.*/ssl/*le.crt; do
    domain=$(echo $d | cut -d/ -f4)
    echo -en "Change: $domain? [yn] "
    read -n 1 answer
    if [ "$answer" != "y" ] ; then
        echo ""
        continue
    fi
    cd $(dirname "${d}")
      # Delete all symlinks in directory
      find . -lname '*' -delete
      # Install new RSA certificate
        /root/.acme.sh/acme.sh --install-cert -d $domain \
        --key-file       /var/www/$domain/ssl/$domain-le.key  \
        --fullchain-file /var/www/$domain/ssl/$domain-le.crt  \
        --reloadcmd     ""
      # Install new ECC certificate
        /root/.acme.sh/acme.sh --install-cert --ecc -d $domain \
        --key-file       /var/www/$domain/ssl/$domain-le_ecc.key  \
        --fullchain-file /var/www/$domain/ssl/$domain-le_ecc.crt  \
        --reloadcmd     ""
 
    echo " Changed: $domain"
done

The result should be real files in /var/www/domain/ssl

Integrate ECC certificates in services
Apache:
Edit master vhost file to:

Code:

SSLCertificateFile <tmpl_var name='document_root'>/ssl/<tmpl_var name='domain'>-le_ecc.crt
SSLCertificateKeyFile <tmpl_var name='document_root'>/ssl/<tmpl_var name='domain'>-le_ecc.key
SSLCertificateFile <tmpl_var name='ssl_crt_file'>
SSLCertificateKeyFile <tmpl_var name='ssl_key_file'>

Postfix
Create new symlinks:

Code:

cd /etc/postfix
ln -sf /var/www/mail.domain.com/ssl/domain-le_ecc.crt smtpd_ecc.cert
ln -sf /var/www/mail.domain.com/ssl/domain-le_ecc.key smtpd_ecc.key
ln -sf /var/www/mail.domain.com/ssl/domain-le.crt smtpd.cert
ln -sf /var/www/mail.domain.com/ssl/domain-le.key smtpd.key

In main.cf

Code:

 # TLS RSA keys path
 smtpd_tls_cert_file = /etc/postfix/smtpd.cert
 smtpd_tls_key_file = /etc/postfix/smtpd.key
 # TLS ECDSA keys path
 smtpd_tls_eccert_file = /etc/postfix/smtpd_ecc.cert
 smtpd_tls_eckey_file = /etc/postfix/smtpd_ecc.key

(The recommended method for new postfix versions is the use of smtpd_tls_chain_files . However, this has problems with the way letsencrypt does ecc certs, so this gives a warning. Therefore I have used the old method. Should be changed in the future)

Dovecot
dovecot.conf

Code:

ssl_cert = </etc/postfix/smtpd_ecc.cert
ssl_key = </etc/postfix/smtpd_ecc.key
ssl_alt_cert=</etc/postfix/smtpd.cert
ssl_alt_key=</etc/postfix/smtpd.key