Securing ISPConfig 3 Control Panel (Port 8080) With Let's Encrypt Free SSL

Discussion in 'Tips/Tricks/Mods' started by ahrasis, Feb 14, 2017.

  1. Why is this better/different from just making a website that uses apache proxy to forward the requests from port 443 (with LE) to port 8080?
  2. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    WOW what a palava!
    Okay so its all good and dandy, mail server just got its own cert! WARNING if like me you lock your ports off on the mail server because you don't actually use them!
    Make sure port 80 and 443 are both open, i had 443 as thats what i thought would be used, but it wasnt.

    Certbot creates a temporary webserver to take in the request (not as clear as it should have been, I presumed that the webserver of the master took in its own request).

    In anycase, DNS validation (yes i modified the script to try that) is difficult to achieve and not needed.

    SOO!! with that now sorted I want to suggest an improvement for your script, updating the firewall and opening a port, then closing it after the cert is created would be a great addition, The script is already running as root to perform the creation so the change and revert to the database should be easy to apply.

    So, when installing a slave server (version 3.2 thats coming up) will this install certbot and then implement a cron job to run your script(or their own implementation)? Im just wondering if this can be automated right from the gate,
    ahrasis likes this.
  3. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    I agree that the script needs some more code to check for port 80 or 443 in order for certbot standalone to work without any of those ports errors.

    For the time being I am helping to fix some bugs in ElkArte forum software for version 1.1.6. I will try to look into it afterwards whenever I am freer.

    In the meantime, feel free to propose your suggested code to the LE4ISPC script at github as well as ISPConfig git for 3.2 master.

    That being said, I now remember the request to add support to neilpang to the merged code in ISPConfig 3.2 master. Hopefully, I have time to read, test and write the additional code for it.
  4. MaxT

    MaxT Active Member HowtoForge Supporter

    not sure if this can be related but also I had some problems with Wheezy, and it seems there is a bug with certbot and Wheezy causing several python errors:

    some people recommends downloading certbot 0.31 release and do some few things:

    # ln -fs /usr/lib/python2.7/plat-x86_64-linux-gnu/ /usr/lib/python2.7/
    # wget
    then editing certbot-auto and do changes around lines 1357 and 1368:
    #pip_version = StrictVersion(check_output([python, '-m', 'pip', '--version'])
    pip_version = StrictVersion(check_output(['pip', '--version'])
    #command = [python, '-m', 'pip', 'install', '--no-index', '--no-deps', '-U']
    command = ['pip', 'install', '--no-index', '--no-deps', '-U']
    and here, many people writes that now just making
    #./certbot-auto --no-self-upgrade
    it will work. Although no in my case. Still some python errors appeared and the ISPC interface didn't preserve the LetsEncrypt option for the websites. However, after doing this for just one domain:
    # ./certbot-auto certonly --no-bootstrap --apache --renew-by-default -d -d
    all works normally again for rest of sites.

    Maybe it can be useful for somebody still using Wheezy
  5. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    I have a shiny New ISPC installed via the (almost) single click method on Debian 9.
    very slick.
    Set up a couple of websites and checked LE and SSL boxes
    websites accessible
    kewl, too easy
    BUT fell at the first hurdle of this tutorial :
    Saving debug log to /var/log/letsencrypt/letsencrypt.log
    Plugins selected: Authenticator standalone, Installer None
    Running pre-hook command: service apache2 stop
    Obtaining a new certificate
    Performing the following challenges:
    http-01 challenge for
    Waiting for verification...
    Cleaning up challenges
    Running post-hook command: service apache2 start
    Hook command "service apache2 start" returned error code 1
    Error output from service:
    Job for apache2.service failed because the control process exited with error code.
    See "systemctl status apache2.service" and "journalctl -xe" for details.
    systemctl status apache2.service
    ● apache2.service - The Apache HTTP Server
       Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
       Active: failed (Result: exit-code) since Mon 2019-03-25 15:57:09 AEDT; 1min 8s ago
      Process: 58200 ExecStop=/usr/sbin/apachectl stop (code=exited, status=1/FAILURE)
      Process: 33939 ExecReload=/usr/sbin/apachectl graceful (code=exited, status=0/SUCCESS)
      Process: 58255 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
     Main PID: 30170 (code=killed, signal=SEGV)
    Mar 25 15:57:09 Hugh2 systemd[1]: Starting The Apache HTTP Server...
    Mar 25 15:57:09 Hugh2 apachectl[58255]: AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:73
    Mar 25 15:57:09 Hugh2 apachectl[58255]: AH00526: Syntax error on line 63 of /etc/apache2/sites-enabled/000-ispconfig.vhost:
    Mar 25 15:57:09 Hugh2 apachectl[58255]: SSLCertificateFile: file '/usr/local/ispconfig/interface/ssl/ispserver.crt' does not exist or is empty
    Mar 25 15:57:09 Hugh2 apachectl[58255]: Action 'start' failed.
    Mar 25 15:57:09 Hugh2 apachectl[58255]: The Apache error log may have more information.
    Mar 25 15:57:09 Hugh2 systemd[1]: apache2.service: Control process exited, code=exited status=1
    Mar 25 15:57:09 Hugh2 systemd[1]: Failed to start The Apache HTTP Server.
    Mar 25 15:57:09 Hugh2 systemd[1]: apache2.service: Unit entered failed state.
    Mar 25 15:57:09 Hugh2 systemd[1]: apache2.service: Failed with result 'exit-code'.
    root@Hugh2:/tmp# service apache2 restart
    Job for apache2.service failed because the control process exited with error code.
    See "systemctl status apache2.service" and "journalctl -xe" for details.
    OK except that
    ls -l /usr/local/ispconfig/interface/ssl/ispserver.crt
    -rwxr-x--- 1 root root 2167 Mar 25 16:11 /usr/local/ispconfig/interface/ssl/ispserver.crt
    I am starting to wonder if there is some strange symbolic link issue.
    I have now read all of this very long thread but I must admit to more than a little confusion.
    Then I found post #272 which to me seemed to imply that just running this script would protect my server.TLD
    So I ran it and it said I already had such a certificate, but I threw caution to the wind and asked for a new one (I mean I get around 7 a week I think) everything seemed to go OK but logging into my https://server.TLD:8080 caused OSX to suggest I had bad certificate.

    The log also contained :
    Server: nginx
    while I am running apache2 although it also stopped and started apache2 not nginx so who knows quite what is going on there.

    I have not tried IMAP yet but if the certificate for the control panel is no good then the one for IMAP will be no good either.

    It seems I have missed something very basic. I mean the references to "read the tutorial" do refer to the instructions at the beginning of this thread don't they ??

    all attempts to help end this confusion gratefully received.

    Oh and yes hostname -f correctly reports server.TLD

  6. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    One further hint is that when I checked the certificate details it was the original self-signed certificate from the original ISPC setup. Oh and running script as per #272 did not ask me any questions. which seemed odd to me.
  7. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    Update:: LE/Certbot has a 5 failure limit in a rolling 7 day period. If you are wary of hitting that limit (I did) then you can simply edit the LEISPC script and look for all the certbot lines: in that line you will want to add a new flag which will send your requests to the test api and your limit is significantly higher. You will of course not get a certificate, just the results to say it worked or it didnt, once its working remove the flags again.
    As i found with the cert the hostname was the issue.
    hostname -f should return like this:
    if it doesnt you will need to edit two files


    after these changes check the command hostname -f again
    Last edited: Mar 26, 2019
  8. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    but as I said above it appears that hostname -f is returning the correct result
    # hostname -f
    but the current certificate that a web browser sees (and complains about) appears to be the self signed (by me) certificate from when ISPC was installed.
    So while the script _may_ have generated a correct certificate (the response from LE seems to indicate this) it does not appear to have put it in the right place
    and I cannot see why.
  9. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    I think browsers can cache the cert, try to hard load the page if you can, in chrome for instance you have a few options, you can open the dev tools/inspector, go to network and click on disable cache. or you can press ctrl and f5 from any page. Not sure it will work but its worth a try, it certainly works when css or javascript files are cached
  10. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    hostname and domain name should be all lowercase.
  11. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    I learned something new, I was not aware of that, but I will bear that in mind in the future.
  12. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    I have been using capitals in hostnames for years.
    Odd it has not bit me in the backside before today.
    I wonder how many things I have now broken in this install.
    Off to do some digging !
    Many thanks
  13. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    as i have now seen a search result, cerbot is case sensitive, I suspect apache is not but without further digging I would not know, to be honest I have always used lower case in all hostnames and domains so it never came up.
  14. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    Will check this out and report back after some sleep
  15. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    Firstly I should address the following
    Just editing /etc/hosts and etc/hostname will not fully alter a servers name _unless_ you either reboot or run the command
    hostname servername
    at least not on my debian 9 server anyway.

    Secondly I should say that yes the sensible thing is to use only lower case hostnames - in fact googling the subject has been quite educational (never too old to learn)

    But thirdly it appears I have just ONE LE certificate left so I need to be careful or cool my heels for a week.

    So I ask this question (which nobody has yet answered) - should simply running the script in post #272 be all I need to do to install the certificate for my ISPC server and its postfix and dovecot ?
    Last edited: Apr 10, 2019
  16. Chris_UK

    Chris_UK Active Member HowtoForge Supporter

    Sorry that we missed that part, we were trying to help you with you host name issue and obviously overlooked it.

    To answer that question. Yes the script creates the required cert and is all that is needed. That is as long as all the services have not been changed manually to point to another certificate in their config files.

    I have just in the past few days run this script on two ubuntu 16.04 server and it worked as expected (save for the hostname issues as you had, I also had those.

    The symbolic links are created in the default locations where you would normally place a certificate and services are restarted.

    One thing to note however, filezilla grumbles about the cert as its for the hostname and not the domain being managed, this is nothing to worry about, just accept the cert and move on. Some mail clients may do that too, for me it didnt happen with mail, because i created a standalone mail server.

    Thank you for the pointer on the hostname servername, I rebooted as it I had updates that I had installed prior to running so I wasnt aware of that step.
  17. Gary Pearce

    Gary Pearce Member HowtoForge Supporter

    Apologies for the late reply - other matters intervened. for those who may follow yes this worked nicely.

    Many thanks
    ahrasis likes this.
  18. In January i've reinstalled my VPS server Ubuntu 16.04 Apache 2 with ISPConfig and added LetsEncrypt certificates to all domains that i have in the same server, as also for my hostname name domain "".
    Letsencrypt was installed following the instructions at:
    Unfortunatelly ISPConfig 3.1 is not renewing automatically...
    In your opinion guys, should I use script "" or i just need to do fix it doing something that maybe you guys could kindly help me.
    My certificate to the ISPConfig is the certificate to my host domain (hostname -f) but the certificates has expired yesterday and nothing was renewed automatically as Tim (ISPConfig Man) always said to me that renews automatically...

    Looking in crontab -l results i have: (nothing regarding renewing the letsencrypt certificates)
    root@vps:/etc/ssl# crontab -l
    * * * * * /usr/local/ispconfig/server/ 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done
    * * * * * /usr/local/ispconfig/server/ 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done

    executing a "ll" command at:
    root@vps:/usr/local/ispconfig/interface/ssl# ll
    total 116
    drwxr-x--- 2 root root 4096 Apr 30 13:49 ./
    drwxr-x--- 9 ispconfig ispconfig 4096 Oct 25 2018 ../
    -rwxr-x--- 1 root root 45 Oct 25 2018 empty.dir*
    lrwxrwxrwx 1 root root 56 Apr 30 13:43 ispserver.crt -> /etc/letsencrypt/live/
    -rwxr-x--- 1 root root 2208 Oct 25 2018 ispserver.crt-181029104028.bak*
    lrwxrwxrwx 1 root root 52 Oct 29 2018 ispserver.crt-181107114627.bak -> /etc/letsencrypt/live/
    lrwxrwxrwx 1 root root 56 Nov 7 11:46 ispserver.crt-190430134249.bak -> /etc/letsencrypt/live/
    lrwxrwxrwx 1 root root 56 Oct 27 2018 ispserver.crt.FOG -> /etc/letsencrypt/live/
    -rwxr-x--- 1 root root 1834 Oct 25 2018 ispserver.csr*
    lrwxrwxrwx 1 root root 54 Apr 30 13:43 ispserver.key -> /etc/letsencrypt/live/
    -rwxr-x--- 1 root root 3243 Oct 25 2018 ispserver.key-181029104028.bak*
    lrwxrwxrwx 1 root root 50 Oct 29 2018 ispserver.key-181107114627.bak -> /etc/letsencrypt/live/
    lrwxrwxrwx 1 root root 54 Nov 7 11:46 ispserver.key-190430134249.bak -> /etc/letsencrypt/live/
    lrwxrwxrwx 1 root root 54 Oct 27 2018 ispserver.key.FOG -> /etc/letsencrypt/live/
    -rwxr-x--- 1 root root 3311 Oct 25 2018*
    -rw------- 1 root root 5274 Apr 30 13:49 ispserver.pem
    -rw------- 1 root root 5451 Oct 27 2018 ispserver.pem-181029104028.bak
    -rw------- 1 root root 7419 Oct 29 2018 ispserver.pem-181107114627.bak
    -rw------- 1 root root 7431 Nov 7 11:46 ispserver.pem-181109182939.bak
    -rw------- 1 root root 7431 Nov 9 18:29 ispserver.pem-181109183038.bak
    -rw------- 1 root root 7431 Nov 9 18:30 ispserver.pem-181112135243.bak
    -rw-r--r-- 1 root root 7192 Jan 6 21:51 ispserver.pem-190106215102.bak
    -rw------- 1 root root 7192 Jan 6 21:51 ispserver.pem-190109100343.bak
    -rw-r--r-- 1 root root 7192 Jan 28 14:47 ispserver.pem-190128144747.bak
    -rw-r--r-- 1 root root 0 Jan 29 13:50 ispserver.pem-190129155033.bak
    -rw------- 1 root root 0 Jan 29 13:50 ispserver.pem-190430134251.bak
    -rw------- 1 root root 5274 Apr 30 13:43 ispserver.pem-190430134846.bak
    -rw------- 1 root root 5274 Apr 30 13:48 ispserver.pem-190430134923.bak

    So going the real directory that is the certificates i have the "ll" resultus: (is that right using archive directory certificates?)
    total 12
    drwxr-xr-x 2 root root 4096 Jan 28 14:47 ./
    drwx------ 14 root root 4096 Mar 11 15:09 ../
    lrwxrwxrwx 1 root root 44 Jan 28 14:47 cert.pem -> ../../archive/
    lrwxrwxrwx 1 root root 45 Jan 28 14:47 chain.pem -> ../../archive/
    lrwxrwxrwx 1 root root 49 Jan 28 14:47 fullchain.pem -> ../../archive/
    lrwxrwxrwx 1 root root 47 Jan 28 14:47 privkey.pem -> ../../archive/
    -rw-r--r-- 1 root root 682 Nov 7 11:26 README

    And executing an "ll" at directory /etc/letsencrypt/archive/ :
    total 76
    drwxr-xr-x 2 root root 4096 Jan 28 14:47 ./
    drwx------ 13 root root 4096 Mar 11 15:09 ../
    -rw-r--r-- 1 root root 2512 Nov 7 11:26 cert1.pem
    -rw-r--r-- 1 root root 2273 Jan 6 21:51 cert2.pem
    -rw-r--r-- 1 root root 2273 Jan 9 10:03 cert3.pem
    -rw-r--r-- 1 root root 1923 Jan 28 14:47 cert4.pem
    -rw-r--r-- 1 root root 1647 Nov 7 11:26 chain1.pem
    -rw-r--r-- 1 root root 1647 Jan 6 21:51 chain2.pem
    -rw-r--r-- 1 root root 1647 Jan 9 10:03 chain3.pem
    -rw-r--r-- 1 root root 1647 Jan 28 14:47 chain4.pem
    -rw-r--r-- 1 root root 4159 Nov 7 11:26 fullchain1.pem
    -rw-r--r-- 1 root root 3920 Jan 6 21:51 fullchain2.pem
    -rw-r--r-- 1 root root 3920 Jan 9 10:03 fullchain3.pem
    -rw-r--r-- 1 root root 3570 Jan 28 14:47 fullchain4.pem
    -rw-r--r-- 1 root root 3272 Nov 7 11:26 privkey1.pem
    -rw-r--r-- 1 root root 3272 Jan 6 21:51 privkey2.pem
    -rw-r--r-- 1 root root 3272 Jan 9 10:03 privkey3.pem
    -rw-r--r-- 1 root root 1704 Jan 28 14:47 privkey4.pem
    Finally when try to acess ISPConfig, Chrome report that this website is using HSTS and do open the panel in fact, as you can see in the screendump below:

    I could try to execute certbot autorenew but TIm already said that is really bad, once it will not work, and i have already paid to prove that is really bad and does not work...
    Considering that i can't access ISPConfig 3.1 Panel...., what should i do....?
    Thanks in advance for any help or a definitively answer to solve it.
    Last edited: Apr 30, 2019
  19. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    Update ISPConfig to the latest git-stable, which has a workaround for a certbot bug that might be causing this. As I understand, you should then disable letsencrypt for this site, save, then enable it again - which isn't as easy to do when you're unable to access the site due to HSTS headers.

    As for manually renewing certificates with certbot, as far as I know the only issue is that after renewal the web server doesn't get restarted, so continues to serve the old certificate - though there may be more to it nowadays, with the aforementioned workaround. When issuing (not renewing) certificates you have more options to mess things up, but "certbot renew" will simply use all the settings saved for each certificate and should be safe to perform .. try with --dry-run first if you want to see what it'll do. If it reissues your certificate, restart the web server and see if you can access the control panel using your server's hostname. If that's good, you should just verify that all server names are included in the certificate which are supposed to be there (because the aforementioned bug did not properly record all the domain names included in the certificate) and disable/enable letsencrypt for that site to regenerate it if needed.
    Fabio IT Consultant likes this.
  20. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    You can access your ISPConfig using its IP address as a workaround and as answered, you can use a workaround added to git-stable for certbot version 30 and above.

    However, I would like to highlight here that if you had setup certbot properly, Let's Encrypt should have emailed you with warning for any inability to renew your certs at least twice right after 60 days i.e. 30 days before the expiry.

    I do think many ignored this, often most did not regularly check their servers for any failures like this and end up complaining only after the 90 days expired.

    It is already announced that in the future, ISPConfig will support Neilpang, another LE client, but though it is quite stable, relying on it alone may have the same consequences as this certbot failure one day.

    So my advise is simple, next time ensure that you have set proper email for your certbot, do regular maintenance of your servers, attend any warning(s) especially of failure to renew LE certs and quickly fix them before the certs expired.

Share This Page