Hi, I know, Let's Encrypt again..... So, I use the ISPC 3.1.6, upgraded from 3.1.5 some times ago. I started to migrate several domains into ISPC, successfully in general. I run into apache config error issue, when I try to enable Let's Encrypt for a domain (it has alias and subdomains, all points to the right server). The essence of the reported error is: The full error message I got into the /var/log/letsencrypt/letsencrypt.log, the following: Code: 2017-07-27 10:28:51,497:DEBUG:certbot.storage:Writing new config /etc/letsencrypt/renewal/fxdeviza.hu.conf. 2017-07-27 10:28:51,499:DEBUG:certbot.reporter:Reporting to user: Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/fxdeviza.hu/fullchain.pem. Your cert will expir e on 2017-10-25. To obtain a new or tweaked version of this certificate in the future, simply run letsencrypt-auto again with the "certonly" option. To non-interactively renew *all* of your certific ates, run "letsencrypt-auto renew" 2017-07-27 10:28:51,546:DEBUG:certbot.reverter:Creating backup of /etc/apache2/sites-available/example.net.vhost-le-ssl.conf 2017-07-27 10:28:51,768:INFO:certbot_apache.configurator:Created an SSL vhost at /etc/apache2/sites-available/example.net.vhost-le-ssl.conf 2017-07-27 10:28:52,470:INFO:certbot_apache.configurator:Deploying Certificate for fxdeviza.hu to VirtualHost /etc/apache2/sites-available/example.net.vhost-le-ssl.conf 2017-07-27 10:28:52,471:INFO:certbot_apache.configurator:Enabling available site: /etc/apache2/sites-available/example.net.vhost-le-ssl.conf 2017-07-27 10:28:52,830:ERROR:certbot.util:Error while running apache2ctl configtest. Action 'configtest' failed. The Apache error log may have more information. AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:72 AH00526: Syntax error on line 86 of /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf: FastCgiExternalServer: redefinition of previously defined class "/var/www/clients/client18/web36/cgi-bin/php5-fcgi-*-80-example.net" 2017-07-27 10:28:52,831:DEBUG:certbot.error_handler:Encountered exception: Traceback (most recent call last): File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 461, in deploy_certificate self.installer.restart() File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1800, in restart self.config_test() File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1823, in config_test raise errors.MisconfigurationError(str(err)) MisconfigurationError: Error while running apache2ctl configtest. Action 'configtest' failed. The Apache error log may have more information. AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:72 AH00526: Syntax error on line 86 of /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf: FastCgiExternalServer: redefinition of previously defined class "/var/www/clients/client18/web36/cgi-bin/php5-fcgi-*-80-example.net" 2017-07-27 10:28:52,831:DEBUG:certbot.error_handler:Calling registered functions 2017-07-27 10:28:52,832:CRITICAL:certbot.client:Rolling back to previous server configuration... 2017-07-27 10:28:53,146:DEBUG:certbot.reporter:Reporting to user: We were unable to install your certificate, however, we successfully restored your server to its prior configuration. 2017-07-27 10:28:53,146:DEBUG:certbot.log:Exiting abnormally: Traceback (most recent call last): File "/root/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module> sys.exit(main()) File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 743, in main return config.func(config, plugins) File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 604, in run _install_cert(config, le_client, domains, new_lineage) File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 469, in _install_cert path_provider.cert_path, path_provider.chain_path, path_provider.fullchain_path) File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 461, in deploy_certificate self.installer.restart() File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1800, in restart self.config_test() File "/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot_apache/configurator.py", line 1823, in config_test raise errors.MisconfigurationError(str(err)) MisconfigurationError: Error while running apache2ctl configtest. Action 'configtest' failed. The Apache error log may have more information. AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:72 AH00526: Syntax error on line 86 of /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf: FastCgiExternalServer: redefinition of previously defined class "/var/www/clients/client18/web36/cgi-bin/php5-fcgi-*-80-example.net" I also checked this via the command line, technically the same: Code: Select the appropriate numbers separated by commas and/or spaces, or leave input blank to select all options shown (Enter 'c' to cancel): 29 Obtaining a new certificate Performing the following challenges: tls-sni-01 challenge for example.net Waiting for verification... Cleaning up challenges Created an SSL vhost at /etc/apache2/sites-available/example.net.vhost-le-ssl.conf Deploying Certificate for example.net to VirtualHost /etc/apache2/sites-available/example.net.vhost-le-ssl.conf Enabling available site: /etc/apache2/sites-available/example.net.vhost-le-ssl.conf Error while running apache2ctl configtest. Action 'configtest' failed. The Apache error log may have more information. AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:72 AH00526: Syntax error on line 78 of /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf: FastCgiExternalServer: redefinition of previously defined class "/var/www/clients/client18/web36/cgi-bin/php5-fcgi-*-80-example.net" Rolling back to previous server configuration... Error while running apache2ctl configtest. Action 'configtest' failed. The Apache error log may have more information. AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:72 AH00526: Syntax error on line 78 of /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf: FastCgiExternalServer: redefinition of previously defined class "/var/www/clients/client18/web36/cgi-bin/php5-fcgi-*-80-example.net" IMPORTANT NOTES: - We were unable to install your certificate, however, we successfully restored your server to its prior configuration. - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/example.net/fullchain.pem. Your cert will expire on 2017-10-25. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew" The following settings are enabled for the domain: SSI, SuExec, Auto-subdomain: www, SSL, PHP-FPM, active. Technically the same as the older and working domains. In fact, it seems it does not only affect one domain, but all of them when I try to enable letsencrypt. Additional info: .htaccess was renamed to avoid sideffects, but this manual step would be nice to have a workaround. Other info: debian 8, tens of working domains, some of them with letsencrypt enabled, others not. Do you have any idea, how to investigate and solve this kind of issue? Thanks, István
Seems as if you manually run the certbot command instead of using the builtin LE SSL functions in ISPConfig. certbot and not ISPConfig created the file /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf which is now causing apache to fial. Delete the file /etc/apache2/sites-enabled/example.net.vhost-le-ssl.conf that certbot created and restart apache.
Thank you for your quick reply Till. Unfortunately the situation is that, first I tried to use ISPConfig, not the CLI. After I got silent refuse of enabling letsencrypt, I went down to get info, what went wrong. Under the /etc/apache2/sites-enabled directory, there are no config files like *-le-ssl.conf. I only have *.vhost files and 000-default.conf, default-ssl.conf, ispconfig.conf Instead, if a domain is working with https (letsencrypt), it has a single vhost file and separate sections for Virtualhost *:80 and *:443 Thanks to your reply, I checked again the working configs, I compared the cases of FastCgiExternalServer lines of *:80 and *:443 sections. In the working config, I have the following in case of *:80 and in case of *:443 Regarding to the original problem in my first post it seems the used definition is really the same as in the http section, which is obviously not good, as it should be So, is it a bug or feature? Is it related to letsencrypt script or to ispconfig?
Info: between the old and new vhost.conf.master there are no significant difference, only proxy related additions.
That's not related to ISPConfig at all, it happened due to you running certbot for a vhost manually instead of using the builtin LE function. certbot modifid the vhost like this by inserting duplicate code. To fix your setup, delete the link to this vhost in the sites-enabled folder, restart apache and then login to ISPConfig and change a value in this website and press save. ispconfig will fix the fauls from certbot then. Never run certbot for an ISPConfig website manually as this must break the site, certbot is not able to modify the vhost in the correct way. Instead of running certbot, enable the LE checkbox in ISPConfig for that site. And in case that tis fails, look at the ispconfig or letsencrypt log to see why LE was not able to issue th cert.
Ok, I understand. I will follow your description. Please note, I only went down using certbot manually AFTER the builtin LE failed.
This does not matter for corrupting your vhost file if you do it before or after trying to use LE in ISPConfig. LE in ISPConfig will fail when LE refuses to issue a cert for this domain or when the domain is unreachable from the server. In case your server is behind a router that prevents that the domain is accessible from the server itself, then disable the LE check under System > server config. But be aware that you have to verify the dns for all domains and subdomains manually then, as LE will not issue a cert for any domain of the site when one of the domains or subdomains of the website is not delegated in DNS.
Ok, the server behind a router, all domains accessible from the internet and LE able to issue the certs, just as was working in 3.1.4. I have several domains working with LE's SSL certificate without problem. Even I have LE certificate for the server name and use it for secure IMAP/POP, too. I also have domain, which is not publicly available, of course, that one cannot get a certificate. So, I have several domains with green browser bar in case of https access. I followed the instruction: logout from ispconfig deleted the symlink of a problematic domain apache restarted logged in to ispconfig checked LE under the site config panel waiting for disappearing the red notification in the top (modification is in progress) -> disappeared there are no new entries in the letsencrypt.log LE is not enabled under the site config panel (unchecked) there are no new entries in ispconfig.log So, I did the following again: logout from ispconfig deleted the symlink and the vhost file itself of a problematic domain apache restarted logged in to ispconfig checked LE under the site config panel waiting for disappearing the red notification in the top (modification is in progress) -> disappeared there are no new entries in the letsencrypt.log LE is not enabled under the site config panel (unchecked) there are no new entries in ispconfig.log BUT the vhost file includes the *:443 section and it is working with https:// correctly.
Like I explained above, you have to disable the LE check under system > server config when your router denies requests to domains hosted on the server itself as ISPConfig can not check the domains when they are unavailable. And unless you do that or configure the router so that allows connects to the domains from the server itself, LE can not work.
Maybe my bad, I was not clear regarding the history of this system, what could help us. When I started using ispconfig some months ago, I put the latest version, which was the 3.1.4 as I remember. I put some domains into the server and set up LE for them, which are still working. After that, I upgraded to 3.1.5 and after that 3.1.6. Nowadays I try to do the same things with new domains as I did in 1st of May, which renewed on 1st of July etc. In the last few days (first I found this about 1-2 weeks before, but I just ignored it, as it was not critical, just an option) I started to solve all this LE issues without success. After I realized, LE remained unchecked for domains, I tried to figure out, what was the reason. Again, the same system was able to get LE certificates without problem and I already have LE enabled domains. DNS managing with the old/new domains are exactly the same, even they are on the same DNS server. So, back to my recent issue, let's say with the domain called example.net, I can see the LE certificates under /etc/letsencrypt/live which means to me that, LE is able to issue the certificate without problem. I have 15 LE certificates on this server. Some domains has LE enabled, some not (like example.net). Reading again the error logs, I found that, the problem is in the vhost config file, where the fastcgiexternalserver defined: inside the virtualhost *:443 section the generated fastcgi filename is the same as in the virtualhost *:80 section. Comparing this error message (because the newly generated vhost config disappear/never exists, because the checkconfig reverse back the previous config) to the earlier working config of an other domain, I can see the difference in the filename: 80 vs 443 Maybe I am wrong, but I think the problem is not the router/NAT/LE itself, but the vhost generation process. Maybe the certbot I use is not fresh or has other issues, I do not know yet. I hope now it more clear what is my situation.
Did you disable the LE check under System > Server config now? If not, then do that, othewise it can't work on NAT router systems.
And regarding vhost file names, ISPConfig never adds the port number to a vhost filename as the ssl and non ssl vhost are in the same file on an ISPConfig server. If you have port numbers in the vhost file names, then these files have been added or modified by LE or you wrote them manually. As soon as you start to use LE / certbot manually, it will destroy the vhost setup of the sites in a way that they can't be managed in ISPConfig anymore and Certbot adds the duplicate line that apache complains about as certbot can't cope with complicated vhost setups, it simply tries to make a copy and change the port, which can't work of course while ispconfig writes two different FastCgiExternalServer lines when writing the file. So the next steps to check for you are if you have any vhost files with ports in their name or any vhosts where http and httpd vhost are not in the same file, such files are not from ISPConfig and will block the system. To fix that, delete the two files of that site (or copy them to a backup directory) and let ispconfig create a new vhost file which will happen when you enable le in that site in ispconfig and press save.
I am afraid, if I disable it, I will lost all my existing and working SSL domains and enable back will not work. As I wrote earlier, about 15 10 domains are working as expected, 5 has LE certificates but has no ssl section in the vhost file and cannot enable in the ispconfig.
Then you will not be able to use ISPConfig for any of these sites anymore until you correct the issues that certbot caused as I outlined above. Sure, and that's the bogus config that you created with certbot and the system will not start working until you have undone the changes that certbot made to the apache config.
There are misunderstanding. Let me bring live example here. I have a domain where LE enabled and working as expected. It created in the beginning, in 1st of May. The config file name is: cbox.palyazatnelkul.hu.vhost The content is attached. Please search these texts in the config file: FastCgiExternalServer /var/www/clients/client3/web15/cgi-bin/php5-fcgi-*-80 FastCgiExternalServer /var/www/clients/client3/web15/cgi-bin/php5-fcgi-*-443 When I got error messages for the new domains, I got the following error message: Please note, I mentioned that, based on this error message, the problem could be the vhost config file generation, where the FastCgiExternalServer defined twice and I wrote, maybe the second line should contains 443 instead of 80. It would be nice to have output of the LE script to know, what is the problem exactly with a domain. Or just put all the output into a logfile for further investigation. Again, I cannot see the reason, why disable LE completely (in system -> config), when I have proved domains with working LE.
That's called debug mode and described in many posts in the forum and the ISPConfig FAQ. But using this mode is not necessary in your case as your problem and it's solution are clear. I described them above. I never told you to disable LE. I told you to disable the LE check, you do this by enabling the checkbox 'Skip Lets Encrypt Check' as LE will not work on a NAT setup when the LE check is enabled. I understand your problem from the first post but I see that you still not get it. Either you may do what I suggested to solve your problem or you won't be able to use ISPConfig for these sites anymore. so it's up to you if you want to solve it or not.
Ok, I got it, not disable LE, but check the checkbox called "Skip Lets Encrypt Check" . Yes, I focused on the error message and I still do not get, how did it work before and does not work now. I give it a try and thank you for your patience
Ok, we just run into scheduled maintenance. So, until LE will not operate again, any kind of cert request useless. https://letsencrypt.status.io/ Update: + I messed up the firewalls on the router and on the server, I just started to restore to the original state
Update: it is working as expected, thanks to TIll. Details: The server is running behind a router/firewall. As Till mentioned patiently to me, check the checkbox called "Skip Lets Encrypt Check" . No further trick was necessary on the router. ConfigServer Security & Firewall is running, but it seems it has nothing to do with LE success, LE will work with running firewall. I had to delete all directories under /usr/local/ispconfig/interface/acme (there was a mkdir error in the debug). Don't forget to check https://letsencrypt.status.io/ And listen to the developer Background: On 7th of July, 2017, there was a commit (completely re-worked Let's Encrypt handling) which should caused that, earlier created domains had valid LE certs, but failed after I upgraded my server. To figure out, why LE failed, I messed with certbot manually in command line -> it was not a good idea. Thank you Till
