ISPCONFIG 3.1.13 + UBUNTU 16.04 + CERTBOT 0.26 upgraded to 0.28 do Apache2 do not start anymore

Hello
Today i've upgraded certbot from version .026 to 0.28 to accomplish compatibility to deprecated TLS-SNI-01 to be replaced by HTTP ou DNS validation....
After that apache2 do no restart anymore...
After trying many things i can't find a way to get it running correctly anymore.
every time that i try to restart apache2 this error come up:
Logs
/var/log/apache2/error.log:
[Mon Jan 28 16:28:28.004671 2019] [ssl:emerg] [pid 14371] AH02311: Fatal error initialising mod_ssl, exiting. See /var/log/ispconfig/httpd/mydomainXXX.com/error.log for more information
AH00016: Configuration Failed
/var/log/ispconfig/httpd/mydomainXXX.com/error.log:
[Mon Jan 28 16:28:28.004419 2019] [ssl:emerg] [pid 14371] AH02572: Failed to configure at least one certificate and key for mydomainXXX.com:443
[Mon Jan 28 16:28:28.004588 2019] [ssl:emerg] [pid 14371] SSL Library Error: error:0906D06CEM routinesEM_read_bio:no start line (Expecting: DH PARAMETERS) -- Bad file contents or format - or even just a forgotten SSLCertificateKeyFile?
[Mon Jan 28 16:28:28.004614 2019] [ssl:emerg] [pid 14371] SSL Library Error: error:0906D06CEM routinesEM_read_bio:no start line (Expecting: EC PARAMETERS) -- Bad file contents or format - or even just a forgotten SSLCertificateKeyFile?
[Mon Jan 28 16:28:28.004658 2019] [ssl:emerg] [pid 14371] SSL Library Error: error:140A80B1:SSL routines:SSL_CTX_check_private_key:no certificate assigned

Versions installed in the Ubuntu 16 server thats is regularly updated:
Server version: Apache/2.4.18 (Ubuntu)
Server built: 2018-06-07T19:43:03

Certbot: 0.28
ISPConfig 3.1.13

Apache do not start anymore showing this error:
myrootuser@server:~# apache2ctl graceful
httpd not running, trying to start
Action 'graceful' failed.
The Apache error log may have more information.

 

dunno the answer to your problem, but I did post a question about the cerbot validation changes, and have been told cerbot with ispconfig doesn't use tls-sni-01, so that shouldn't be a problem. no idea whether using anything before certbot 0.28 will be a problem, (I currently have 0.23), i'm prepared to wait till feb 13/14 to find out since the certs should still work anyway.
so you may be able to revert your certbot version.

only other thing I can think of currently is maybe a newer version of openssl is required?

 

Tim
Pretty much honestly, i've made everything that an well experienced professional would do.
But the problem has begun after done the following instructions: from https://community.letsencrypt.org/t/how-to-stop-using-tls-sni-01-with-certbot/83210
""
Let’s Encrypt is removing support for domain validation with TLS-SNI-01. If you’re using Certbot and received an email titled “Action required: Let’s Encrypt certificate renewals”, here’s how to fix the problem. It’s possible you’ve upgraded Certbot in the time since the last TLS-SNI validation mentioned in the email, in which case you’re fine. These instructions tell you how to check.

1. Confirm your Certbot version is 0.28 or higher:
   
   certbot --version || /path/to/certbot-auto --version
   

If the version is less than 0.28, you need to upgrade your Certbot. Visit https://certbot.eff.org/ 5.9k and follow the instructions for your webserver and OS.

1. Remove any explicit references to tls-sni-01 in your renewal configuration:
   
   sudo sh -c "sed -i.bak -e 's/^\(pref_challs.*\)tls-sni-01\(.*\)/\1http-01\2/g' /etc/letsencrypt/renewal/*; rm -f /etc/letsencrypt/renewal/*.bak"
   
   
2. Do a full renewal dry run:
   
   sudo certbot renew --dry-run
   

If the dry run succeeds, and your Certbot version is 0.28 or higher, you’re good to go! If it fails, fix the validation problems you see and try again.

If you get a connection refused or connection timeout, you may have a firewall blocking port 80. tls-sni-01 used port 443, but http-01 uses port 80. Ideally your web server should allow both ports. If that’s not possible, for instance because your ISP blocks port 80, you’ll need to switch to the dns-01 challenge, or use an ACME client that supports tls-alpn-01.

Note: if you installed Certbot in late 2015 or early 2016, it may be called letsencrypt or letsencrypt-auto (the project was renamed). Follow the instructions at https://certbot.eff.org 2.0k to install the latest version.

Credit to @_az for the suggestion to write more step-by-step instructions and @jsha for rewriting these instructions with that suggestion in mind.

We done the update version above once we received the email below from [email protected]
"Hello,
Action may be required to prevent your Let's Encrypt certificate renewals
from breaking.
If you already received a similar e-mail, this one contains updated
information.
Your Let's Encrypt client used ACME TLS-SNI-01 domain validation to issue
a certificate in the past 60 days. Below is a list of names and IP
addresses validated (max of one per account):
mydomain.com.br (xxx.yyy.zzz.uuu) on 2019-01-09
TLS-SNI-01 validation is reaching end-of-life. It will stop working
temporarily on February 13th, 2019, and permanently on March 13th, 2019.
Any certificates issued before then will continue to work for 90 days
after their issuance date.

You need to update your ACME client to use an alternative validation
method (HTTP-01, DNS-01 or TLS-ALPN-01) before this date or your
certificate renewals will break and existing certificates will start to
expire.

Our staging environment already has TLS-SNI-01 disabled, so if you'd like
to test whether your system will work after February 13, you can run
against staging: https://letsencrypt.org/docs/staging-environment/

If you're a Certbot user, you can find more information here:
https://community.letsencrypt.org/t/how-to-stop-using-tls-sni-01-with-certbot/83210

Our forum has many threads on this topic. Please search to see if your
question has been answered, then open a new thread if it has not:
https://community.letsencrypt.org/

For more information about the TLS-SNI-01 end-of-life please see our API
announcement:
https://community.letsencrypt.org/t/february-13-2019-end-of-life-for-all-tls-sni-01-validation-support/74209

Thank you,
Let's Encrypt Staff
"

 

Once i follow the best pratices that i've spread out among many IT Technicians that worked in my teams, and also once i've always enforced and used deeply ITIL since 1999 working for UK internacional companies...i fortunately had backups from /etc and a full backup of my server...
So to uncommit the changes, i've recover the /etc tar.gz and renamed at /etc/apache2/sites-available the mydomain.vhost file and after that the apache2 came up back, finally.
I've realized that certbot 0.28 goes intro the vhost files from the domains that i had renewed the certificates directly from certbot cli commands, and this new version changed the paths certificates from /www/clients.... that are in fact ln links to path directly to /etc/letsencrypt/live/xxxdomais certificates...
I've deactivated the sites and turned off SSL and Letsencrypt by unchecking the regarding checkboxes of those domains...after that, reactivated and renabled the SSL and Letsencrypt checkboxes....to keep the domains on complaince to ISPConfig vhost format again, what i really prefer, to do not disrupte ISPConfig features and future updates or changes by the ispconfig panel, that are a real benefit from what you guys perfectly done....ISPConfig is really fabulous....i love it.
So now i have a few trick questions:
1) Should i go back to previous version 0.26 or can i keep the version 0.28?
If i keep using 0.28 the renew process always produce an alert line in red color saying that:

Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for xxxxxxx.com.br
TLS-SNI-01 is deprecated, and will stop working soon. (THIS LINE IS RED COLOR IN THE TERMINAL)
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/apache2/sites-enabled/XXXX.com.br​

2) Should i change letsencrypt to do not use TLS-SNI-01 challenge anymore or do nothing?
3) What will happen at February 13th?
Comment: The letsencrypt warning email said ""
TLS-SNI-01 validation is reaching end-of-life. It will stop working
temporarily on February 13th, 2019, and permanently on March 13th, 2019.
Any certificates issued before then will continue to work for 90 days
after their issuance date."
4) ISPConfig renew crontab script suggestions will keep running automatically considering the conditions above?
5) Curiously i have many sites running with path and certificates generated by ISP panel as also have other sites running well too with certificates using /etc/letsencrypt path and certificates renewed by certbot. OH My GOD...Linux are really clever!
6) Last one: Why in /etc/apache2/sites-available there is some files with both xxxdomain.com.br { .vhost and .conf} and others ones with just xxxdomain.com.br.vhost? What should really be there, both or just .vhost files?

Thanks in advance to you Till and our forum contributors.

 

i would like to keep using the ISPConfig method that you said is webroot...where should i change it to do not use TNS anymore, something that i've bet that came up after certbot updated to version 0.28.

 

understood, that text message is from certbot cli command to renew certificates....but increadibly both methods are running on your perfect server Till... you guys are really clever...Congratulations...this ISPConfig can solve things that are not even predicted by you guys.

 

This instructions caused the problem indeed...lol...i really do not want to use those instructions in fact...otherwise the problem will come up again...if just in case you agree with my conclusion and perspective about it...lol

 

"I wonder why you got tls-sni-01, I don't have ...."
I am quite sure that the TNS-SL-01 came up as effect of following instructions or after i've faced any error, some forum suggested to goes to letsencrypt.conf or directory /etc/letsencrypt/renewal WHEN I'VE INSTALLED THE SERVER...but i use ISPCONFIG for more than 4 years...
Looking into two different domain .conf files that i have in /etc/letsencrypt/renewal, what is the best pratice or the correct format to be 100% compliant to ISP:
1) "# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/vps.xxxdomain.com.br
cert = /etc/letsencrypt/live/vps.xxxdomain.com.br/cert.pem
privkey = /etc/letsencrypt/live/vps.xxxdomain.com.br/privkey.pem
chain = /etc/letsencrypt/live/vps.xxxdomain.com.br/chain.pem
fullchain = /etc/letsencrypt/live/vps.xxxdomain.com.br/fullchain.pem

# Options used in the renewal process
[renewalparams]
server = https://acme-v02.api.letsencrypt.org/directory
account = xxxxxxxxyyyyy
authenticator = webroot
[[webroot_map]]
vps.xxxdomain.com.br = /var/www/html
"
or

2)
# renew_before_expiry = 30 days
version = 0.28.0
archive_dir = /etc/letsencrypt/archive/yyydomain.com.br
cert = /etc/letsencrypt/live/yyydomain.com.br/cert.pem
privkey = /etc/letsencrypt/live/yyydomain.com.br/privkey.pem
chain = /etc/letsencrypt/live/yyydomain.com.br/chain.pem
fullchain = /etc/letsencrypt/live/yyydomain.com.br/fullchain.pem

# Options used in the renewal process
[renewalparams]
server = https://acme-v02.api.letsencrypt.org/directory
authenticator = apache
account = yyyyyyyyyyyyyy
installer = apache

or perhaps a third format LIKE I HAVE ALSO...ps: I've never change it....i do not have the habitual to change standard things...i know thta it can cause real mess.
3)
# renew_before_expiry = 30 days
version = 0.26.1
archive_dir = /etc/letsencrypt/archive/zzzdomain.com.br
cert = /etc/letsencrypt/live/zzzdomain.com.br/cert.pem
privkey = /etc/letsencrypt/live/zzzdomain.com.br/privkey.pem
chain = /etc/letsencrypt/live/zzzdomain.com.br/chain.pem
fullchain = /etc/letsencrypt/live/zzzdomain.com.br/fullchain.pem

# Options used in the renewal process
[renewalparams]
account = zzzzzzzzzzzzzzz
server = https://acme-v01.api.letsencrypt.org/directory
authenticator = webroot
rsa_key_size = 4096
post_hook = echo '1' > /usr/local/ispconfig/server/le.restart
[[webroot_map]]
zzzdomain.com.br = /usr/local/ispconfig/interface/acme
www.zzzdomain.com.br = /usr/local/ispconfig/interface/acme

 

In fact i have domains using different PHP methods, fastcgi, hhvm, etc...it explains why it is happening, many thanks Till...

 

i really researching the web to find instructions to disable TLS-SN-01 and return to http challenge method.....but just in case you know where to change it, please let me know...
Thanks a lot in advance.

 

I've already changed it as you said.....pointing all domains to ....../acme
added posthook also
and commented installer line
and changed authenticator = webroot for the domains that had "apache"

 

Morning Till
Could you please read the questions above and reply to me those that you could contribute with your thought about...