Configuring ISPConfig to work with ondrej's co-installable PHP 5.6 and 7.0 package on Ubuntu

Discussion in 'Installation/Configuration' started by cbj4074, Feb 23, 2016.

  1. cbj4074

    cbj4074 Member

    I'm using the Ubuntu PPA available at https://launchpad.net/~ondrej/+archive/ubuntu/php to run PHP 5.6 and 7.0 side-by-side, while I establish my migration-path for users.

    I have both PHP versions working side-by-side beautifully (PHP 5.6 as the primary and 7.0 as a separate "PHP Version"), with one exception: I don't seem to be able to set an explicit socket file location for the PHP version that I add as a separate "PHP Version".

    The primary PHP version provides the opportunity to enter a directory in which to store socket files. Why not provide the same option for the additional PHP version? It's as though that field is simply missing. Maybe there's a good reason for that.

    Fundamentally, the problem seems to be that if I switch the PHP version pull-down for the website from 5.6 to 7.0 or vice-versa, the target version tries to reload its daemon and the socket-file still exists from the origin version, the daemon for which is still running. Or something similar. This results in a 503 at the gateway layer, until I stop/start both PHP daemons in the correct order. If there is some better/simpler way to write the socket files for 5.6 and 7.0 to different directories?

    Thanks for any help!
     
  2. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    Last edited: Feb 23, 2016
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    The socket directory is systemwide for all php versions, there is no need to provide a separate socket directory or to create any sockets as that's all handled by ISPConfig automatically. Your issue is not related to the socket directory.

    The problem that you describe happens when PHP 7 is used on a system with systemd support (e.g. Debian 8) but has no systemd starter file or it has a starter file and an init script but has been started trough the init script and not the systemd unit. When your system uses systemd, then ensure that your custom PHP version has a systemd unit file and is only started and stopped through that systemd unit and not trough a possibly existing init script. The reason for that is that php fails to load the new config when you have started it with a init script but ispconfig uses the correct way trough systemd to reload it.
     
    madmucho likes this.
  4. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    Thank you till, that could be the issue we have in the other topic - uhm just one question... What to put into the field for "Path to init script" - Howto Debian 8 + PHP 7 installs init + systemd files and systemd is used to stop/start on boot
     
  5. till

    till Super Moderator Staff Member ISPConfig Developer

    Debian 8 supports systemd and init files, so there can be systems with and without systemd support. When you follow the tutorial then you have both options available but you should only use the one that is correct for your server. so if your system has systemd installed, then only use systemd. If your system has no systemd, then use the init script. All is fine as long as you don't mix the usage of the init systems as their is no conflict in their existance.

    On systemd systems, the name of the init script is the same as the name of the systemd unit. so ispconfig simply uses the filename from this field as systemd unit name.

    The setup in the above tutorial works fine, I use it on several servers. You just get in trouble when you run e.g.:

    /etc/init.d/php-7.0.3-fpm

    run on a system that uses systemd instead of the correct command:

    systemctl start php-7.0.3-fpm.service
     
    madmucho and ztk.me like this.
  6. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    Thank you for the heads up :) First time using systemd and didn't think that could be a big deal ... I'll fiddle around with that later and try to check wether I can get that 503 error again or not. I have some trust it won't appear, thanks.
     
  7. cbj4074

    cbj4074 Member

    Hehe, no worries, ztk.me, on the pool directory != socket directory. I appreciate your quick reply all the same.

    In my case, the system in question is not running systemd; it's using init scripts.

    Hmm... I hate to ask, but do you mind explaining a bit more about how ISPConfig handles this automatically?

    For example, assume that PHP 5.6 and PHP 7.0 share the same socket directory, and I create a new site that is using PHP 5.6 (the default version in ISPConfig). Everything is running fine. At this point in time, the website's PHP socket file exists at /var/run/php/web1.sock.

    Then I choose to create an Additional PHP version for 7.0, and I set the website in question to use the 7.0 version.

    When I do this, won't the 7.0 daemon attempt to create a socket file at the exact same location that the 5.0 daemon is using, /var/run/php/web1.sock?
     
  8. ztk.me

    ztk.me Well-Known Member HowtoForge Supporter

    When switching php versions, the old php-config will be deleted, the old daemon gets reloaded, new config will be written and new daemon will be reloaded - basically no issue there if you don't mix up initd and systemd like me :)
    Well, didn't test it under load yet - there's a little sleep(1) in the codes

    Overlapping socket files were my first thought, too when initially seeing error 503 and solved it after restarting both php-fpm. But till sounds as he would know what he's doing :rolleyes::D
     
  9. cbj4074

    cbj4074 Member

    I attempted to configure another server with co-installable PHP versions, as we've been discussing, and I encountered the exact same problem: whenever I switch a given website from one PHP version to the other, it results in a 503 error at the web-server because PHP-FPM fails to reload. From what I'm able to determine, the reason is as I stated above: socket file overlap.

    @ztk.me, it sounds as though you've examined the relevant source-code (presumably /usr/local/ispconfig/server/plugins-available/nginx_plugin.inc.php near line 2478), and that there is indeed some logic intended to prevent this problem, but it doesn't seem to be working as intended in my case. That said, it could very well be a configuration error on my part. (I'll take a closer look at the code later.)

    I'll do a bit more digging and attempt to provide a reproducible test-case, but for now, the only measure that fixes this for me is to use a different socket directory for each PHP version. And the only means by which I'm able to do that is by modifying the pool configuration files directly. Obviously, this approach does not work long-term, because anytime ISPConfig modifies the pool configuration file, my changes will be overwritten.

    I don't see anything in the php_fpm_pool.conf template that would enable me to distinguish between PHP versions.

    Given that each PHP version's details are already stored in $php_version, I don't see any good reason not to add something like $php_version['php_fpm_socket_dir'] and allow the administrator to configure this path for each PHP version. This seems like a cleaner way to approach the problem than to call sleep(1) or fight against other possible race-conditions.

    Would I be incorrect to state that the current approach guarantees at least 1 second of an unreachable website due to the delay between cleaning-up the old socket and creating the new one?
     
  10. till

    till Super Moderator Staff Member ISPConfig Developer

    I just tested it here on a Debian 8 perfect server, switched 10 times force and back between PHP 5.6 and 7 and there was no issue. I use the same setup on all my live servers and did not had an issue there was well yet.
     
  11. cbj4074

    cbj4074 Member

    Thank you for taking the time to test this on your own setup, Till. Clearly, the issue is on my end. I'll post an update if/when I figure out what's happening!
     
  12. till

    till Super Moderator Staff Member ISPConfig Developer

  13. cbj4074

    cbj4074 Member

    It is entirely possible that there is some difference in the pool configuration that is responsible for the quirky behavior that I observe.

    But given that a significant number of ISPConfig users are likely to use this Ubuntu PPA, with increasing frequency as PHP 7 gains traction and people begin to migrate, this issue will likely resurface. I'd like to find a solution before it becomes a widespread problem.

    So, I'm seeing several issues here. The first issue is that ISPConfig is using incorrect PID file locations for the PHP-FPM daemons. When I enable debug logging in ISPConfig, I see entries like this (when switching between PHP versions):

    restarting php-fpm: /sbin/start-stop-daemon --stop --signal USR2 --quiet --pidfile /var/run/php5-fpm.pid --name php5-fpm

    That PID file location is incorrect. (Does the --name value need to be "php5.6-fpm", too?) The actual PID file location is /run/php/php5.6-fpm.pid (note the 5.6).

    I can change this path in the pool configuration, but I would prefer not to (because future Ubuntu package updates will complain about a conflict), especially if there is a means by which to change the PID path that ISPConfig uses. But it looks as though the path is hard-coded in /usr/local/ispconfig/server/mods-available/web_module.inc.php on line 251. And the line is question is wrapped in Ubuntu-14.04-only conditional logic.

    This raises another question: where is the PID file for PHP 7.0 (or any "Additional PHP Version") defined? Maybe defining it is not necessary?

    In an effort to at least get this working, even if temporarily, I changed the PID file path in the PHP 5.6 pool configuration to match the value that ISPConfig is expecting.

    Starting fresh, I have the 5.6 daemon and the 7.0 daemons running simultaneously. They are sharing a single socket directory. The test website is set to use PHP 5.6, which is the default version as defined in ISPConfig.

    When I change the PHP version to 7.0, via the drop-down menu in the GUI, here's what I see in the debug log:

    Code:
    25.02.2016-16:19 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    25.02.2016-16:19 - DEBUG - Found 1 changes, starting update process.
    25.02.2016-16:19 - DEBUG - Calling function 'ssl' from plugin 'nginx_plugin' raised by event 'web_domain_update'.
    25.02.2016-16:19 - DEBUG - Calling function 'update' from plugin 'nginx_plugin' raised by event 'web_domain_update'.
    25.02.2016-16:19 - DEBUG - SSL Disabled.
    25.02.2016-16:19 - DEBUG - Writing the vhost file: /etc/nginx/sites-available/platform-postgres.localhost.com.vhost
    25.02.2016-16:19 - DEBUG - Writing the PHP-FPM config file: /etc/php/7.0/fpm/pool.d/web6.conf
    25.02.2016-16:19 - DEBUG - Removed PHP-FPM config file: /etc/php/5.6/fpm/pool.d/web6.conf
    25.02.2016-16:19 - DEBUG - Calling function 'restartPHP_FPM' from module 'web_module'.
    25.02.2016-16:19 - DEBUG - Restarting php-fpm: /sbin/start-stop-daemon --stop --signal USR2 --quiet --pidfile /var/run/php5-fpm.pid --name php5-fpm
    25.02.2016-16:19 - DEBUG - Calling function 'restartPHP_FPM' from module 'web_module'.
    25.02.2016-16:19 - DEBUG - Restarting php-fpm: service php7.0-fpm reload
    25.02.2016-16:19 - DEBUG - nginx status is: running
    25.02.2016-16:19 - DEBUG - Calling function 'restartHttpd' from module 'web_module'.
    25.02.2016-16:19 - DEBUG - Restarting httpd: service nginx restart
    25.02.2016-16:19 - DEBUG - nginx restart return value is: 0
    25.02.2016-16:19 - DEBUG - nginx online status after restart is: running
    25.02.2016-16:19 - DEBUG - Processed datalog_id 119
    25.02.2016-16:19 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
    
    At this point, when I hit my test page, which simply outputs phpinfo(), I still get PHP 5.6. And I'm trying to determine why.

    Oddly enough, nothing is written to either PHP-FPM log (/var/log/php5.6-fpm.log or /var/log/php7.0-fpm.log) when the above debug entries are written to ISPConfig's log. I had expected to see some chatter in relation to the daemons being reloaded.

    If I "service php7.0-fpm reload", nothing changes. And, again, there is no log entry for this.

    But if I "service php7.0-fpm restart", I get this:

    Code:
    [25-Feb-2016 16:01:17] ERROR: An another FPM instance seems to already listen on /run/php/web6.sock
    [25-Feb-2016 16:01:17] ERROR: FPM initialization failed
    
    This seems to indicate that PHP 5.6 never actually relinquished the socket file when restartPHP_FPM() was called. Does this indicate a problem with that "/sbin/start-stop-daemon..." command?

    UPDATE:

    Echoing the exit status from this command, if I run it manually, yields "1", which, according to http://man7.org/linux/man-pages/man8/start-stop-daemon.8.html , means "If --oknodo was not specified and nothing was done." So, nothing was done. Why not? Well, removing the "--quiet" switch reveals the reason: "No php5-fpm found running; none killed."

    I've tried a number of different values for the "--name" argument, e.g. "php5.6-fpm", and they all return the same message.

    Yet, if I simply remove the --name switch entirely, everything works! Wow. What value does the --name switch have, then? Can it be removed from the command safely?

    In summary, this boils down to two issues:

    1.) It would be nice to be able to specify the PID file location for each version of PHP.

    2.) Is there any reason to keep the --name switch on this hard-coded command that applies only to Ubuntu 14.04?

    Thanks!
     
    Last edited: Feb 26, 2016
  14. cbj4074

    cbj4074 Member

    Actually, simply commenting-out lines 250-252 solves the entire problem.

    Perhaps that conditional logic could be improved so that it doesn't affect PHP packages that don't suffer from the issue it was intended to address. I'll think on that. :)

    UPDATE: Created an issue for this at https://git.ispconfig.org/ispconfig/ispconfig3/issues/3789
     
    Last edited: Mar 2, 2016

Share This Page