ispconfig_update.sh quits without error

Hello folks,

I´m facing a problem whenever I tried to update my ispconfig 3 server as follows:
root@web01:/var/log/ispconfig# /etc/init.d/cron stop
[ ok ] Stopping periodic command scheduler: cron.
root@web01:/var/log/ispconfig# ispconfig_update.sh


--------------------------------------------------------------------------------
Cool ASCII LOGO HERE
--------------------------------------------------------------------------------

>> Update

Please choose the update method. For production systems select 'stable'.
WARNING: The update from GIT is only for development systems and may break your current setup. Do not use the GIT version on servers that host any live websites!
Note: Update all slave server, before you update master server.

Select update method (stable,git) [stable]: stable

--2016-02-18 11:35:22-- http://www.ispconfig.org/downloads/ISPConfig-3-stable.tar.gz
Resolving www.ispconfig.org (www.ispconfig.org)... 104.25.55.32, 104.25.54.32
Connecting to www.ispconfig.org (www.ispconfig.org)|104.25.55.32|:80... connected.

--- lots of mamboojamboo downloads links stuff and all - no errors.

>> Update

Operating System: Debian 7.0 (Wheezy/Sid) or compatible
This application will update ISPConfig 3 on your server.
Shall the script create a ISPConfig backup in /var/backup/ now? (yes,no) [yes]: yes

Creating backup of "/usr/local/ispconfig" directory...
Creating backup of "/etc" directory...
root@web01:/var/log/ispconfig#

And thats it. No log, no ERRORS. ISPCONFIG already in Debug mode. Nothing in the logs. I dunno what´s going on.

I noticed that after this attempt, my cron is stucking in server.sh. Nothing is being updated anymore. I have to manually kill server.sh process and remove the lock file, runs in the shell the server.sh (no errors too) - but in cron it runs only the first time - than it hangs again.

This is the master server. Im trying to update him first.
If someone had already faced anything similar to this, please HALP!

Basic outputs:
/var/log/ispconfig/ispconfig.log
18.02.2016-11:32 - DEBUG - There is already an instance of server.php running. Exiting.
18.02.2016-11:33 - DEBUG - There is already an instance of server.php running. Exiting.

root@web01:/var/log/ispconfig# wget -q -O htf-common-issues.php "http://gitplace.net/pixcept/ispconfig-tools/raw/stable/htf-common-issues.php" && php -q htf-common-issues.php

Fatal error: Call to undefined function mysqli_connect() in /var/log/ispconfig/htf-common-issues.php on line 184

About the stucked process of server.sh:
root@we01:/var/log/ispconfig# /usr/local/ispconfig/server/server.sh > /dev/null >> /var/log/ispconfig/cron.log
18.02.2016-11:45 - DEBUG - There is already an instance of server.php running. Exiting.

root@web01:/var/log/ispconfig# ps aux|grep server.sh
root 19945 0.0 0.0 10720 1332 ? Ss 11:40 0:00 /bin/sh -c /usr/local/ispconfig/server/server.sh > /dev/null >> /var/log/ispconfig/cron.log
root 19947 0.0 0.0 10736 1528 ? S 11:40 0:00 /bin/sh /usr/local/ispconfig/server/server.sh

root@webservices01:/var/log/ispconfig# kill -9 19945 19947
root@webservices01:/var/log/ispconfig# rm -f /usr/local/ispconfig/server/temp/.ispconfig_lock
root@webservices01:/var/log/ispconfig# /usr/local/ispconfig/server/server.sh > /dev/null >> /var/log/ispconfig/cron.log
18.02.2016-11:47 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
18.02.2016-11:47 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
root@webservices01:/var/log/ispconfig# /usr/local/ispconfig/server/server.sh > /dev/null >> /var/log/ispconfig/cron.log
18.02.2016-11:47 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
18.02.2016-11:47 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock


I have runned it several times from shell just like that to make sure that nothing in server.sh is braking the removal of the lock file or crashing the code. It stucks only in crontab. :/
So weird hm?

 

It wasn´t, Till

root@web01:/var/log/ispconfig# which mysql
/usr/bin/mysql



Also, the version of deb.pack php/mysql installed in the system:

root@web01:/var/log/ispconfig# dpkg -l|grep mysql
ii dovecot-mysql 1:2.1.7-7+deb7u1 amd64 MySQL support for Dovecot
ii libclass-dbi-mysql-perl 1.00-3 all extensions to Class:BI for MySQL
ii libdbd-mysql-perl 4.021-1+b1 amd64 Perl5 database interface to the MySQL database
ii libmysqlclient-dev 5.5.47-0+deb7u1 amd64 MySQL database development files
ii libmysqlclient18:amd64 5.5.47-0+deb7u1 amd64 MySQL database client library
ii libtime-piece-mysql-perl 0.06-2 all module adding MySQL-specific methods to Time:iece
ii mysql-client 5.5.47-0+deb7u1 all MySQL database client (metapackage depending on the latest version)
ii mysql-client-5.5 5.5.47-0+deb7u1 amd64 MySQL database client binaries
ii mysql-common 5.5.47-0+deb7u1 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii mysql-server 5.5.47-0+deb7u1 all MySQL database server (metapackage depending on the latest version)
ii mysql-server-5.5 5.5.47-0+deb7u1 amd64 MySQL database server binaries and system database setup
ii mysql-server-core-5.5 5.5.47-0+deb7u1 amd64 MySQL database server binaries
ii php-mdb2-driver-mysql 1.5.0b2-1 all PHP PEAR module to provide a MySQL driver for MDB2
ii php5-mysql 5.4.45-0+deb7u2 amd64 MySQL module for php5
ii postfix-mysql 2.9.6-2 amd64 MySQL map support for Postfix
ii pure-ftpd-mysql 1.0.36-1.1 amd64 Secure and efficient FTP server with MySQL user authentication
ii roundcube-mysql 0.7.2-9+deb7u1 all metapackage providing MySQL dependencies for RoundCube

 

root@web01:~# which php
/usr/local/bin/php


And also posted that libs were installed not by compilling... but thru "apt" during multiserver perfect-setup-howto.

ii php-mdb2-driver-mysql 1.5.0b2-1 all PHP PEAR module to provide a MySQL driver for MDB2
ii php5-mysql 5.4.45-0+deb7u2 amd64 MySQL module for php5

It was from apt during the setup of the server. The server was working perfectly till my attempt of upgrade thru the script. All jobs were being processed within 1 minute as crontab says. After the crash of the update script, my jobs stopped to being processed saying that another server.php is being executed. Stucked lockfiles and all... ... I have to constantly remove lock file and kill the server.php process by hand.

If I run the crontab line in the shell .. everything works. No matter how fast I repeat the command.

If crontab runs it automatically, it hangs after the second or third time. Dunno what the heck is happening, Till ... it is very weird - I never got a behavior like that in ANY ispconfig install - Im new to the forum but Im a heavy user of ispconfig. 5 different clients of mine are using it in a "big-setup-multiserver" enterprise and never had any problems. (congrats, by the way. )

I´m still logged in trying to fetch more informations about that strange error. My spectations is that - whenever I´m abble to do the upgrade, this JOB-Stucked problem will go away.

I´ll build up a test server with the old packages that I have.. and try to upgrade it. Them compare all .deb relative to ispconfig, php and mysql.. see if I can find anything different - I dunno ... Im so lost here, man .. :-/

Thank you for your attention and care, by the way.
D:.

 

Just noticed that something is calling up server.sh script at the same time crond it is...


root 3078 0.0 0.0 10720 1332 ? Ss 19:00 0:00 /bin/sh -c /usr/local/ispconfig/server/server.sh > /dev/null >> /var/log/ispconfig/cron.log
root 3080 0.0 0.0 10736 1532 ? S 19:00 0:00 /bin/sh /usr/local/ispconfig/server/server.sh

I dunno from where this is comming from. Crontab has only one line (the first one in grep with the dev null pointer)

 

which php tells you /usr/local/bin/php
I assume there's something with your PATH ENV.
Running update.php from your shell will use /usr/local/bin/php which probably is missing mysql(i) support

/usr/local/ispconfig/server/server.sh sets PATHS correctly and uses
/usr/bin/php -q /usr/local/ispconfig/server/server.php
to call php which is most likely system installed php with mysql support

 

I have managed to reproduce whats hanging my JOBS with server.sh:

root@web01:/etc/bind# /usr/bin/php -q /usr/local/ispconfig/server/server.php
19.02.2016-22:55 - DEBUG - Set Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock
df: `/var/www/clients/client1/web6/log (deleted)': No such file or directory
postqueue: warning: Mail system is down -- accessing queue directly

- and stucks right there. Strange that - this server isnt a postfix server - it is only webhost in my multiserver setup. theres never being a postfix instalation into that....

 

Ok I think I have confused you guys - thats two separate things. The first problem - that is the origin of this post is that ispconfig_update.sh crashes without any errors. -

I belive that - if php was configured wrongly or compiled in a way that "ispconfig" vars couldn´t detect it, nothing whould have worked before - my customers, the webmail, other aps, ispconfig main panel and actions.. etc. This server was working perfectly since Feb 2015 (production time - Install time was Jan 2015).

Everything works - only the ispconfig_update.sh script that does not. Thats the main topic here.

Now let´s see my php path´s:

root@web01:/usr/local/bin# which php
/usr/local/bin/php

This is the current profile env var for php.

into /usr/local/bin I have:
root@web01:/usr/local/bin# ls -l
total 52372
lrwxrwxrwx 1 root staff 51 Sep 10 2014 ispconfig_patch -> /usr/local/ispconfig/server/scripts/ispconfig_patch
lrwxrwxrwx 1 root staff 55 Jan 14 2015 ispconfig_update_from_dev.sh -> /usr/local/ispconfig/server/scripts/ispconfig_update.sh
lrwxrwxrwx 1 root staff 55 Jan 14 2015 ispconfig_update.sh -> /usr/local/ispconfig/server/scripts/ispconfig_update.sh
-rwxr-xr-x 1 root staff 817 Feb 16 19:44 pear
-rwxr-xr-x 1 root staff 838 Feb 16 19:44 peardev
-rwxr-xr-x 1 root staff 754 Feb 16 19:44 pecl
lrwxrwxrwx 1 root staff 24 Mar 26 2015 phar -> /usr/local/bin/phar.phar
-rwxr-xr-x 1 root staff 53488 Mar 26 2015 phar.phar
-rwxr-xr-x 1 root staff 26802134 Mar 26 2015 php
-rwxr-xr-x 1 root staff 26732432 Mar 26 2015 php-cgi
-rwxr-xr-x 1 root staff 2143 Mar 26 2015 php-config
-rwxr-xr-x 1 root staff 4522 Mar 26 2015 phpize
-rwxr--r-- 1 getmail root 497 Jan 14 2015 run-getmail.sh

Checking the place of ispconfig_update.sh

root@web01:/usr/local/bin# whereis ispconfig_update.sh
ispconfig_update: /usr/local/bin/ispconfig_update.sh

Now - the script of server.sh - uses a different php that my original env. I dont know why.
/usr/bin/php -q /usr/local/ispconfig/server/server.php

This /usr/bin/php is a link to /etc/alternatives/php that is a link to /usr/bin/php5

So I have tried to use fullpath of php in ispconfig_update.sh ... but none of them changes anything. After /etc backup, script crashes without any errors:

/usr/local/bin/php -q /usr/local/ispconfig/server/scripts/ispconfig_update.php
or
/usr/bin/php -q /usr/local/ispconfig/server/scripts/ispconfig_update.php


same "silent" crash.

 

Maneged to run the htf-common-issues.php by donwloading it and running with /usr/bin/php

vi htf_report.txt

##### RUNNING SERVER PROCESSES #####

[INFO] I found the following web server(s):
Apache 2 (PID 350)
[WARN] I could not determine which mail server is running.
[WARN] I could not determine which pop3 server is running.
[WARN] I could not determine which imap server is running.
[INFO] I found the following ftp server(s):
PureFTP (PID 15938)

##### LISTENING PORTS #####
(only ()
Local (Address)
[localhost]:10024 (4256/amavisd-new)
[anywhere]:3306 (2652/mysqld)
[localhost]:11211 (3757/memcached)
[anywhere]:111 (27098/rpcbind)
[anywhere]:2255 (22964/sshd)
[localhost]:783 (4139/spamd.pid)
[anywhere]:21 (15938/pure-ftpd)
***.***.***.***:53 (651/named)
***.***.***.***:53 (651/named)
[localhost]:53 (651/named)
[localhost]:953 (651/named)
[anywhere]:39777 (2763/rpc.statd)
[anywhere]:10050 (29866/zabbix_agentd)
*:*:*:*::*:40072 (2763/rpc.statd)
[localhost]11 (27098/rpcbind)
*:*:*:*::*:2255 (22964/sshd)
*:*:*:*::*:8080 (350/apache2)
*:*:*:*::*:80 (350/apache2)
*:*:*:*::*:8081 (350/apache2)
*:*:*:*::*:21 (15938/pure-ftpd)
*:*:*:*::*:53 (651/named)
*:*:*:*::*:953 (651/named)
*:*:*:*::*:443 (350/apache2)
[localhost]0050 (29866/zabbix_agentd)




##### IPTABLES #####
Chain INPUT (policy ACCEPT)
target prot opt source destination
fail2ban-ssh tcp -- [anywhere]/0 [anywhere]/0 multiport dports 22

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain fail2ban-dovecot-pop3imap (0 references)
target prot opt source destination

Chain fail2ban-sasl (0 references)
target prot opt source destination

Chain fail2ban-ssh (1 references)
target prot opt source destination
RETURN all -- [anywhere]/0 [anywhere]/0

 

That solved the problem of the update Till. Thank you for your patience and help. Still - I dunno how I got a php compilled in a install from "Debian Perfect Server" How To :/ thats weird. But thank you a lot! Now I'll hope that cron starts to work with services.sh script properly again to process my jobs. Thank you!

 

This will also happen if you upgrade to PHP 7, because PHP 7 does not include the legacy "mysql" extension, which the updater requires, apparently. This is not a "bug", because ISPConfig 3.0.* does not support PHP 7.

If you're planning to upgrade to PHP 7, be sure to install all of the updates to ISPConfig first. Once ISPConfig 3.1 is generally-available and stable, you can upgrade to that, in which case this is no longer an issue.

Another alternative is to have a separate PHP 5 version installed on the same server, and then you can use it to update ISPConfig, even when you're running PHP 7 for other virtual-hosts on the same server.

I have confirmed that this approach works. In my particular case, I'm using co-installable PHP 7.0 and PHP 5.6. The package defines 7.0 as the default, so to be able to update ISPConfig, it's necessary to instruct the various ISPConfig scripts involved to use PHP 5.6.

I had to specify the full path to the 5.6 executable in the following files:

1.) /usr/local/ispconfig/server/server.sh

Code:

#!/bin/sh

PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin

. /etc/profile

umask 022

if [ -f /usr/local/ispconfig/server/lib/php.ini ]; then
  PHPINIOWNER=`stat -c %U /usr/local/ispconfig/server/lib/php.ini`
  if [ $PHPINIOWNER == 'root' ] || [ $PHPINIOWNER == 'ispconfig'  ]; then
  export PHPRC=/usr/local/ispconfig/server/lib
  fi
fi

cd /usr/local/ispconfig/server
/usr/bin/php5.6 -q /usr/local/ispconfig/server/server.php

cd /usr/local/ispconfig/security
/usr/bin/php5.6 -q /usr/local/ispconfig/security/check.php

2.) /usr/local/ispconfig/server/scripts/ispconfig_update.sh

Code:

#!/bin/bash

/usr/bin/php5.6 -q /usr/local/ispconfig/server/scripts/ispconfig_update.php

3.) /usr/local/ispconfig/server/scripts/update_from_tgz.sh

Code:

#!/bin/bash

cd /tmp

if [ -f ISPConfig-3-stable.tar.gz ]
then
rm -f ISPConfig-3-stable.tar.gz
fi

wget http://www.ispconfig.org/downloads/ISPConfig-3-stable.tar.gz
if [ -f ISPConfig-3-stable.tar.gz ]
then
   tar xvfz ISPConfig-3-stable.tar.gz
   cd ispconfig3_install/install/
   /usr/bin/php5.6 -q update.php
   rm -rf /tmp/ispconfig3_install/install
   rm -f ISPConfig-3-stable.tar.gz
else
   echo "Unable to download the update."
fi

exit 0

Now, I'm able to execute "ispconfig_update.sh" and the script no longer terminates immediately after backing-up the configuration; the update succeeds as normal.