Sites Set with PHP-FPM - Subs still FCGI - And LE issues

Discussion in 'Installation/Configuration' started by Viper_iii, Dec 4, 2018.

  1. Viper_iii

    Viper_iii Member

    Does that make sense?
    Domain supersite.com (fake domain name for example)
    PHP-FPM - 7.2
    Addiitonal PHPs all set and running - verify php is correct and running.
    used this for PHP setup https://www.howtoforge.com/tutorial...fig-3-from-debian-packages-on-debian-8-and-9/
    no crons so should be all good..

    subdomain No Redirect - was previously set for L Redirect
    blog.supersite.com
    keeps redirecting to
    blog.supersite.com/php-fcgi/index.php - which is bad
    remote the extra junk and the page loads ...

    /etc/apache2/sites-available/domain.tld.vhost
    I can see redirects

    Code:
                    RewriteEngine on
                    RewriteCond %{REQUEST_URI} ^/\.well-known/acme-challenge/
                    RewriteRule ^ - [END]
                    RewriteCond %{HTTP_HOST}   ^supersite\.co$ [NC]
    
                    RewriteRule   ^/(.*)$ http://www.supersite.com  [L]
    
                    RewriteCond %{HTTP_HOST}   ^www\.supersite\.co$ [NC]
    
                    RewriteRule   ^/(.*)$ http://www.supersite.com  [L]
    
                    RewriteCond %{HTTP_HOST}   ^blog\.supersite\.com$ [NC]
                                    RewriteCond %{REQUEST_URI} !^/webdav/
                    RewriteCond %{REQUEST_URI} !^/php-fcgi/
                    RewriteCond %{REQUEST_URI} !^/blog/
    
                    RewriteRule   ^/(.*)$ /blog/$1  [L]
    
                    RewriteCond %{HTTP_HOST}   ^blog\.supersite\.co$ [NC]
                                    RewriteCond %{REQUEST_URI} !^/webdav/
                    RewriteCond %{REQUEST_URI} !^/php-fcgi/
                    RewriteCond %{REQUEST_URI} !^/blog/
    
                    RewriteRule   ^/(.*)$ /blog/$1  [L]
    
                    RewriteCond %{HTTP_HOST}   ^www\.blog\.supersite\.co$ [NC]
                                    RewriteCond %{REQUEST_URI} !^/webdav/
                    RewriteCond %{REQUEST_URI} !^/php-fcgi/
                    RewriteCond %{REQUEST_URI} !^/blog/
    
                    RewriteRule   ^/(.*)$ /blog/$1  [L]


    so setting the site to php-fcgi should fix in general correct?
    Put Sub Redirect back to L...
    and appears to work... but checking cache - had it working in chrome and was due to caching non-redirected path..



    ----
    Lets Encrypt..
    I think a few years ago I manually set a certificate from CF (Cloudflare) and messed the autocert process up...
    currently unable to delete the certificate and just install an LE for the supersite.com domain which would also cover blog.supersite.com as well if checkbox was left unchecked.... I think.... on Subdomain page.....
     
  2. Viper_iii

    Viper_iii Member

    now after playing with it the http://www.supersite.com is getting errors
    whoops... had http > https set at cloudflare and on ispconfig Rewrite HTTP to HTTPS box checked

    all disabled currently and still getting issue.
    ERR_TOO_MANY_REDIRECTS

    another issue is SSL - unable to delete existing cert... can type random chars and save and they stay - but unable to delete
     
  3. Viper_iii

    Viper_iii Member

    found too many more issues... job que not processing due to IDS PHP errors with 7.0.11 which shoudln't even be installed I don't think...

    basically not taking any changes in the interface so disable SSL isn't working still enabled just causing issues now.

    [INTERFACE]: PHP IDS Alert.Total impact: 15<br/> Affected tags: dt, id, lfi<br/> <br/> Variable: POST.php_fpm_init_script | Value: /etc/init.d/php-7.0.11
     
  4. Viper_iii

    Viper_iii Member

    blew it away and went back to a backup... - safe to ignore this thread...

    deleted all PHP installs again and reloaded based on
    https://www.howtoforge.com/tutorial...fig-3-from-debian-packages-on-debian-8-and-9/
    still had to add several packages that are not in there like mbstrings / xml / and one or two others to get the admin console back working
    instead of just adding for 5.6 I added for all versions just so its same/same

    set the server to run default of 5.6 for its main PHP.

    currently just fighting LE - Server was upgraded from 3.0 and 3.1 early versions with LE addons -
    don't see LE skip option in Systems > config > Web
    do have /var/log/letsencrypt and is executing just getting errors:

    Code:
    2018-12-05 19:46:03,965:DEBUG:<userrenamed>:Received <Response [200]>. Headers: {'Content-Length': '658', 'Expires': 'Wed, 05 Dec 2018 19:46:03 GMT', 'Strict-Transport-Security': 'max-age=604800', 'Server': 'nginx', 'Connection': 'keep-alive', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Wed, 05 Dec 2018 19:46:03 GMT', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json'}. Content: '{\n  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",\n  "meta": {\n    "caaIdentities": [\n      "letsencrypt.org"\n    ],\n    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",\n    "website": "https://letsencrypt.org"\n  },\n  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",\n  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",\n  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",\n  "randomstring": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",\n  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"\n}'
    2018-12-05 19:46:03,966:DEBUG:acme.client:Received response <Response [200]> (headers: {'Content-Length': '658', 'Expires': 'Wed, 05 Dec 2018 19:46:03 GMT', 'Strict-Transport-Security': 'max-age=604800', 'Server': 'nginx', 'Connection': 'keep-alive', 'Pragma': 'no-cache', 'Cache-Control': 'max-age=0, no-cache, no-store', 'Date': 'Wed, 05 Dec 2018 19:46:03 GMT', 'X-Frame-Options': 'DENY', 'Content-Type': 'application/json'}): '{\n  "keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",\n  "meta": {\n    "caaIdentities": [\n      "letsencrypt.org"\n    ],\n    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",\n    "website": "https://letsencrypt.org"\n  },\n  "newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",\n  "newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",\n  "newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",\n  "randomstring": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",\n  "revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"\n}'
    2018-12-05 19:46:03,968:DEBUG:certbot.main:Exiting abnormally:
    Traceback (most recent call last):
      File "/<userrenamed>/.local/share/letsencrypt/bin/letsencrypt", line 11, in <module>
        sys.exit(main())
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 693, in main
        return config.func(config, plugins)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 504, in obtain_cert
        le_client = _init_le_client(config, auth, installer)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 356, in _init_le_client
        acc, acme = _determine_account(config)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/main.py", line 341, in _determine_account
        config, account_storage, tos_cb=_tos_cb)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 119, in register
        regr = perform_registration(acme, config)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/certbot/client.py", line 149, in perform_registration
        return acme.register(messages.NewRegistration.from_data(email=config.email))
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/acme/client.py", line 98, in register
        response = self.net.post(self.directory[new_reg], new_reg)
      File "/<userrenamed>/.local/share/letsencrypt/local/lib/python2.7/site-packages/acme/messages.py", line 160, in __getitem__
        raise KeyError('Directory field not found')
    KeyError: 'Directory field not found'
    so is it a python issue?
    not sure how to re-load / update - reconfigure - when update script stops because version is current... reading more on that
     
  5. Viper_iii

    Viper_iii Member

    found Skip SSL settings
    system > Config > Web > SSL > Skip LE Check (finnally found it...) buried doh!

    now re-attempting - no new errors generated but log didn't change at all... so far... Monitoring
     
  6. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    On debian I'd uninstall certbot/letsencrypt and any related packages, and remove any version you manually installed (try "which certbot" to see where it's running from, and remove that /<userrenamed>/.local/ directory from your error, etc.) then install certbot from stretch-backports.
     
  7. Viper_iii

    Viper_iii Member

    Ok - will give that a shot...
    think it was manually installed and not even apt-get install but not sure...
    yup was manual back in 2016...
    just hadn't updated much I guess... probably borked it months ago and didn't notice... - yup was using cloudflare ssl .. so yup - never even checked it!

    got it installed setup the jessie-backport and have it started but now have errors like:

    The client lacks sufficient authorization :: Invalid response from http://sitename
    what is odd is the site name... its one of alias domains for the site..

    DNS problem: NXDOMAIN looking up A for blog.shortname.co
    vs blog.domainname.com which probably errored futher up I'm guessing...
     
  8. till

    till Super Moderator Staff Member ISPConfig Developer

    The LE check (which you disabled as part of the tests) removes such unreachable domains from SSL automatically as LE will not issue a cert otherwise for any domain of the site if one domain is unreachable.
     
  9. Viper_iii

    Viper_iii Member

    You both were correct...

    Reloaded LE
    Also deleted old one..
    Read your other reload older post.
    Re- downloaded an ran update.php per post.
    Resync executed

    Removed alias domains from LE except www alias
    Removed / no redirect www from blog.domain
    LE log shows it created at least no errors..

    Enabled http-https redirect loop
    Do have the many phps install and is set at 7.2
    Sites are static so expect error in config.
    blog.domain is cms not doing https redirect there... Loaded fine before https force...

    CFlare https rules disabled

    _--------++++_
    Any chance we'll see external DNS options? For LE cert generation?
    For an idea of generation and gui setup - pfse see plugin since 2.2/2.3 acme has had it...
    If I knew how to do scripting better I'd have that unit generate mine then scp the pem and key to hosts and have host run update script weekly to see if file changed and update if different...
    But unfortunately I suck with that!!!!!
    & ISPsconfig apparently!
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    ISPConfig uses domain based auth and that works pretty well, you just have to ensure that your domain is reachable from outside so that LE can connect to it. And in case that you use some fancy redirect rules that prevent LE from accessing it's token, then just add some exclude rules so that the LE path is not rewritten. DNS based auth for LE is required for wildcard certs only and has many other issues on it's own, so don't think that DNS auth is the holy grail and will prevent all LE failures.
     
  11. Viper_iii

    Viper_iii Member

    DNS external auth vs DNS local using dns apis & Unrelated to my issue with redirects - yes understood -

    Primary reason is I have sites internal only & others like the panel on different ports - not this server other than 8080 which 8443 would be helpful and I can modify that...

    Looking back I gave up on LE before 3.1 and just use self signed + cloudflare so it proxies CF SSL if desired...
    Problem is only when trying to use SSL in ISPCP the way I have it messed up...
    Not doing redirects that I can remember... But it is ~2015 when it was first installed and not anything major... Can move the sites and db associated in 1-2 hours.

    Have fought redirect loop before and is most likely php related and I'm good at messing up!

    Only redirects are auto generated I don't believe i edited vhosts / added any options (options empty)

    Hope any of that made sense
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

    LE in ISPConfig is really easy and straightforward in ISPConfig, I use it on many sites and have not a single failure. All that LE needs is that it can access a simple text file on your server, so really easy, straightforward and not a source of errors unless you have a misconfigured server or missed adding DNS records. And ispconfig excludes the LE paths automatically in rules that you create trough ISPConfig, which makes it even easier.

    Testing LE is really simple:

    1) Create a test token on the shell with:
    touch /usr/local/ispconfig/interface/acme/.well-known/acme-challenge/test.txt

    2) Now test all domains and subdomains that you want to get an LE cert for with a web browser:

    http://yourdomain.tld/.well-known/acme-challenge/test.txt

    this file must be reachable so you don't get a 404. If that's the case. then LE will work. and as a site node, so not enable http to https redirect in a site that has no https as this must lead to an endless loop. So first enable ssl and LE (or ssl any any other cert) and then enable http to https redirect.
     
  13. till

    till Super Moderator Staff Member ISPConfig Developer

    And then check the LE log if your initial python error is resolved as you can't get an LE cert when certbot is not able to run on your server.
     
  14. Viper_iii

    Viper_iii Member

    Yes LE works... And when 80/443 are enabled publicly no issue...

    And I have it working now ...
    Now is issues with borked php / vhosts

    I'll try to sanitize my output and post
    Expect you / others will see obvious red flags...

    External DNS is pretty much on all sites on this unit..

    Reading through panel LE cert later - that might have the info on CF API plugin I'm after... Might not..
     
  15. Viper_iii

    Viper_iii Member

    Ok yup need some idea of WTF I have messed up..

    built a brand new install and no extra php and still having redirect issues only when i enable HTTPS for sites.

    interesting on a fresh box I setup also had an issue where I did https://panel.site.com and it loaded a site but not sure what site as it was a WP install but there was only 3 sites setup and all WP installs were complete... so have no Idea where that default https:// was aimed when no sites had it enabled...

    does that make sense?!?

    Anyway..

    sanitized output... doing that now..
    Old Server - Odd redirect issues:
    apachectl -S
    Code:
    all domains are Cloudflare Name Servers and use that for DNS externally
    
    AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:73
    VirtualHost configuration:
    *:8081                 host.mainsite.com (/etc/apache2/sites-enabled/000-apps.vhost:9)
    *:8080                 host.mainsite.com (/etc/apache2/sites-enabled/000-ispconfig.vhost:9)
    *:80                   is a NameVirtualHost
             default server host.mainsite.com (/etc/apache2/sites-enabled/000-default.conf:1)
             port 80 namevhost host.mainsite.com (/etc/apache2/sites-enabled/000-default.conf:1)
             port 80 namevhost associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:7)
                     alias www.associationmain.com
                     alias www.associationoldtld.com
                     alias associationoldtld.com
             port 80 namevhost mainsite.com (/etc/apache2/sites-enabled/100-mainsite.com.vhost:7)
                     alias www.mainsite.com
                     alias www.mainsitealias.co
                     alias mainsitealias.co
                     alias blog.mainsite.com
                     alias blog.mainsitealias.co
             port 80 namevhost offroadsite.com (/etc/apache2/sites-enabled/100-offroadsite.com.vhost:7)
                     alias www.offroadsite.com
    *:443                  is a NameVirtualHost
             default server associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:128)
             port 443 namevhost associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:128)
                     alias www.associationmain.com
                     alias www.associationoldtld.com
                     alias associationoldtld.com
             port 443 namevhost offroadsite.com (/etc/apache2/sites-enabled/100-offroadsite.com.vhost:117)
                     alias www.offroadsite.com
    ServerRoot: "/etc/apache2"
    Main DocumentRoot: "/var/www/html"
    Main ErrorLog: "/var/log/apache2/error.log"
    Mutex rewrite-map: using_defaults
    Mutex authdigest-client: using_defaults
    Mutex fcgid-proctbl: using_defaults
    Mutex ssl-stapling: using_defaults
    Mutex ssl-cache: using_defaults
    Mutex default: dir="/var/lock/apache2" mechanism=fcntl
    Mutex mpm-accept: using_defaults
    Mutex fcgid-pipe: using_defaults
    Mutex authdigest-opaque: using_defaults
    Mutex watchdog-callback: using_defaults
    PidFile: "/var/run/apache2/apache2.pid"
    Define: DUMP_VHOSTS
    Define: DUMP_RUN_CFG
    Define: ENABLE_USR_LIB_CGI_BIN
    User: name="www-data" id=33
    Group: name="www-data" id=33
    
    New Server - Same Odd redirect issues:
    apachectl -S
    Code:
    AH00548: NameVirtualHost has no effect and will be removed in the next release /etc/apache2/sites-enabled/000-ispconfig.conf:73
    VirtualHost configuration:
    *:8081                 cp.mainsite.com (/etc/apache2/sites-enabled/000-apps.vhost:9)
    *:8443                 cp.mainsite.com (/etc/apache2/sites-enabled/000-ispconfig.vhost:9)
    *:80                   is a NameVirtualHost
             default server cp.mainsite.com (/etc/apache2/sites-enabled/000-default.conf:1)
             port 80 namevhost cp.mainsite.com (/etc/apache2/sites-enabled/000-default.conf:1)
             port 80 namevhost associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:7)
                     alias www.associationmain.com
                     alias www.associationoldtld.com
                     alias associationoldtld.com
             port 80 namevhost mainsite.com (/etc/apache2/sites-enabled/100-mainsite.com.vhost:7)
                     alias www.mainsite.com
                     alias blog.mainsite.com
                     alias www.mainsitealias.co
                     alias mainsitealias.co
             port 80 namevhost offroadsite.com (/etc/apache2/sites-enabled/100-offroadsite.com.vhost:7)
                     alias www.offroadsite.com
    *:443                  is a NameVirtualHost
             default server associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:120)
             port 443 namevhost associationmain.com (/etc/apache2/sites-enabled/100-associationmain.com.vhost:120)
                     alias www.associationmain.com
                     alias www.associationoldtld.com
                     alias associationoldtld.com
             port 443 namevhost mainsite.com (/etc/apache2/sites-enabled/100-mainsite.com.vhost:119)
                     alias www.mainsite.com
                     alias blog.mainsite.com
                     alias www.mainsitealias.co
                     alias mainsitealias.co
             port 443 namevhost offroadsite.com (/etc/apache2/sites-enabled/100-offroadsite.com.vhost:117)
                     alias www.offroadsite.com
    ServerRoot: "/etc/apache2"
    Main DocumentRoot: "/var/www/html"
    Main ErrorLog: "/var/log/apache2/error.log"
    Mutex watchdog-callback: using_defaults
    Mutex rewrite-map: using_defaults
    Mutex ssl-stapling-refresh: using_defaults
    Mutex authdigest-client: using_defaults
    Mutex fcgid-proctbl: using_defaults
    Mutex ssl-stapling: using_defaults
    Mutex proxy: using_defaults
    Mutex ssl-cache: using_defaults
    Mutex default: dir="/var/run/apache2/" mechanism=default
    Mutex mpm-accept: using_defaults
    Mutex fcgid-pipe: using_defaults
    Mutex authdigest-opaque: using_defaults
    PidFile: "/var/run/apache2/apache2.pid"
    Define: DUMP_VHOSTS
    Define: DUMP_RUN_CFG
    Define: ENABLE_USR_LIB_CGI_BIN
    User: name="www-data" id=33
    Group: name="www-data" id=33
    
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    When there is no vhost on a specific IP and port combination where the server name matches the requested server name, then apache will show the first vhost that it finds in it's configuration instead as it has no better matches. That's the default behavior of an apache server and not ISPConfig specific and that's also no redirect, it's simply an emergency fallback of apache when it tries to deliver a request when there is no matching vhost. So when you request https://b.tld and there is no site with domain b.tld which has SSL enabled, then apache must show the content of the first site it find which has SSL enabled on the same IP address. If you don't want that apache shows a site of a customer in that case, create a default vhost for that purpose. The default vhost must be first in alphabet, so you choose a domain name for it that is always first like 000default.tld. The domain name does not has to exist, but you must enable SSL for it which means you will have to create a self-signed SSL certificate. This site will then catch all non-matching requests for your server. Of course, it will produce an SSL error, but this is inevitable. So the better alternatives are: a) Enable SSL for all sites or b) use two IP addresses, one for sites with SSL and one for sites without SSL and ensure that apache does not listen on port 443 on the IP for sites that shall not be reachable by ssl.
     
  17. Viper_iii

    Viper_iii Member

    ok that makes sense...

    not ispcp / issue but default behavior of default site and ssl not being enabled.

    the part that doesn't make sense is the redirect issue / redirect loop.

    appears to be resolved by removing the checkbox in redirect:
    http > https
    at least in testing on the new box this is the case...
    trying to follow the redirect to understand why its looping when that is enabled vs no...
    basically if you force it to redirect it errors - turn it off you have to force via another method like server options - which is the same rule that should be done by the checkbox correct?
     
  18. till

    till Super Moderator Staff Member ISPConfig Developer

    Regarding redirect loop, is the website empty or is there a cms installed in it? Most cms incl. WordPress do such redirects on their own, so when you enable the redirect in ispconfig and you have e.g. WordPress installed and the site URL in WordPress has http:// instead of https://, then you will get a redirect loop as apache redirects from http to https and WordPress back from https to http. The same can happen when you have a conflicting rule in a .htaccess file or so which redirects back from https to http.

    And the checkbox simply adds an apache redirect rule "if not https then redirect the current URL to https" as you can see in the vhost, so it should not matter if you use the checkbox or add the redirect rule in a .htaccess file or in the apache directives.
     
  19. Viper_iii

    Viper_iii Member

    on some sites there were options so the checkbox created a double redirect loop - cleaned those up.

    the main site though was static content - no cms - and was having the issue...
    seems to be better now on the old one -

    Found that something was off with php7.1 engine was getting
    FPM instance seems to already listen on socket
    unable to restart that or any php version so rebooted and all got better - magic...

    from there enabled HTTPS from cloudflare strict as LetsEncrypt certs were active - (could use CF cert as well though or even self signed in front of CF cert and get SSL as well)
    just wanted to get consistent LE Certs going
    still if I enable redirect to HTTPS locally either checkbox / Options it get redirect loop on static pages
    blog.mainsite.com/info.php also gets redirect loop so not CMS related - even though it will be if I force that to https in the CMS as well...

    mainsite.com - .htaccess is blank so no rewrite in there... forgot to verify that as well.


    only issue now is on older one setting the panel to 8443 fails to work after running update - says all good but not active on 8443....
     

Share This Page