Somthing segfaults when server.php is run, and server.php never ends to run, the lockfile is there and the process is running. Excerp from log file: Code: Apr 4 11:38:01 ic CRON[18409]: (root) CMD (/usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) Apr 4 11:38:01 ic CRON[18408]: (root) CMD (/usr/local/ispconfig/server/server.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done) Apr 4 11:38:14 ic kernel: [13906.912572] php[18392]: segfault at 7fffca8d9f80 ip 000055b77ce7c241 sp 00007fffca8d9f80 error 6 in php7.3[55b77cc82000+261000] Apr 4 11:38:14 ic kernel: [13907.128543] Code: ff 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 57 41 56 49 89 d6 41 55 49 89 f5 41 54 49 89 fc 53 48 81 ec f8 00 00 00 48 8b 45 18 <48> 89 8d e0 fe ff ff 4c 89 85 f0 fe ff ff 44 89 8d 18 ff ff ff 48 Processes running Any tips how to trouble shoot this?
... and this could also be helpful, seems to be line 22 Code: Sun 04 Apr 2021 11:45:01 AM CEST 04.04.2021-11:45 - WARNING - There is already a lockfile set, but no process running with this pid (18982). Continuing. Sun 04 Apr 2021 11:46:01 AM CEST 04.04.2021-11:46 - WARNING - There is already an instance of server.php running with pid 19035. Sun 04 Apr 2021 11:46:10 AM CEST /usr/local/ispconfig/server/server.sh: line 22: 19035 Segmentation fault /usr/bin/php -q -d disable_classes= -d disable_functions= -d open_basedir= /usr/local/ispconfig/server/server.php Sun 04 Apr 2021 11:47:01 AM CEST 04.04.2021-11:47 - WARNING - There is already a lockfile set, but no process running with this pid (19035). Continuing. Sun 04 Apr 2021 11:48:02 AM CEST 04.04.2021-11:48 - WARNING - There is already an instance of server.php running with pid 19249. Sun 04 Apr 2021 11:48:17 AM CEST /usr/local/ispconfig/server/server.sh: line 22: 19249 Segmentation fault /usr/bin/php -q -d disable_classes= -d disable_functions= -d open_basedir= /usr/local/ispconfig/server/server.php Sun 04 Apr 2021 11:49:01 AM CEST 04.04.2021-11:49 - WARNING - There is already a lockfile set, but no process running with this pid (19249). Continuing.
Well, not so helpful, the line referes to the call Code: cd /usr/local/ispconfig/server /usr/bin/php -q \ -d disable_classes= \ -d disable_functions= \ -d open_basedir= \ /usr/local/ispconfig/server/server.php
Which PHP Version do you use? Are the PHP packages the ones that ship with the OS or sury, or did you build PHP yourself? Do you use any custom PHP modules e.g. ones manually compiled with pear? Any other segfaults in the systems syslog file? A segfault can also be hardware related like faulty RAM, motherboard issues etc. Do you have enough RAM?
Also can come from filesystem corruption, you might run fsck and force reinstall all php related packages
Its Debian 10, php 7.3. auto-install script used, hence sury used. No special php modules Code: root@ic:/var/log# php -m [PHP Modules] calendar Core ctype curl date dom exif fileinfo filter ftp gd gettext hash iconv imagick imap intl ionCube Loader json libxml mbstring mysqli mysqlnd openssl pcntl pcre PDO pdo_mysql pdo_sqlite Phar posix pspell readline recode Reflection session shmop SimpleXML soap sockets sodium SPL sqlite3 standard sysvmsg sysvsem sysvshm tidy tokenizer wddx xml xmlreader xmlrpc xmlwriter xsl Zend OPcache zip zlib [Zend Modules] Zend OPcache the ionCube PHP Loader + ionCube24 No other segfaults in syslog. RAM is 4GB for a non-production server, with a couple of sites, should be no worries. Its a VPS, hence no MB etc. issues.
I found out how to increment the queue counter, so there is no queue items pending now. No segfaults either when server.sh is run. Will now test different queue jobs to see what happens. It segfaults for some kind of queue job.
I enabled debug logging, added a mail domain alias, and here is where it hangs (and segfaults I assume) Code: 05.04.2021-16:23 - DEBUG - Unable to register function 'process' from plugin 'software_update_plugin' for event 'software_update_inst_insert' 05.04.2021-16:23 - DEBUG - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. 05.04.2021-16:23 - DEBUG - Found 1 changes, starting update process. 05.04.2021-16:23 - DEBUG - Calling function 'user_settings_update' from plugin 'rspamd_plugin' raised by event 'mail_forwarding_insert'. q.sh: line 24: 17680 Segmentation fault /usr/bin/php -d disable_classes= -d disable_functions= -d open_basedir= /usr/local/ispconfig/server/server.php Anyone?
There is a bug in 3.2.3 creating a loop writing spamfilter settings for a domain alias, make sure you have updated to the current version (3.2.4). I didn't see that causing segfaults, and don't know that it would without an outside problem (hardware or whatever), but maybe you issue will clear with that?
Oki, I am in 323. Will update and see. Since it is a multi server environment, should I update I any particular order? And, I did test this on a slave mail server, and there was no issues in this server. Both servers are identical, except one is master and the other mail only. Code: Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Unable to register function 'process' from plugin 'software_update_plugin' for event 'software_update_inst_insert' Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Found 1 changes, starting update process. Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Replicated from master: REPLACE INTO `mail_forwarding` (`forwarding_id`,`sys_userid`,`sys_groupid`,`sys_perm_user`,`sys_perm_group`,`sys_perm_other`,`server_id`,`source`,`destination`,`type`,`active`,`allow_send_as`,`greylisting`) VALUES ('6','1','30','riud','riud','','5','@foo.com','@bar,com','aliasdomain','y','n','n') Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Calling function 'user_settings_update' from plugin 'rspamd_plugin' raised by event 'mail_forwarding_insert'. Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Processed datalog_id 3530 Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Calling function 'restartRspamd' from module 'mail_module'. Mon 05 Apr 2021 04:41:01 PM CEST 05.04.2021-16:41 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock Mon 05 Apr 2021 04:41:01 PM CEST finished server.php. Mon 05 Apr 2021 04:42:01 PM CEST 05.04.2021-16:42 - DEBUG - Unable to register function 'process' from plugin 'software_update_plugin' for event 'software_update_inst_insert' Mon 05 Apr 2021 04:42:01 PM CEST 05.04.2021-16:42 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock Mon 05 Apr 2021 04:42:02 PM CEST finished server.php. Mon 05 Apr 2021 04:43:01 PM CEST 05.04.2021-16:43 - DEBUG - Unable to register function 'process' from plugin 'software_update_plugin' for event 'software_update_inst_insert' Mon 05 Apr 2021 04:43:01 PM CEST 05.04.2021-16:43 - DEBUG - Remove Lock: /usr/local/ispconfig/server/temp/.ispconfig_lock Mon 05 Apr 2021 04:43:01 PM CEST finished server.php.
I update the master first, then all slave servers, and for any version updates always answer yes to reconfigure permissions in the master db. There is a setting to put servers in a maintenance mode first; I don't know exactly what that does (disables changes in the ui?), but I pretty much never (remember to) use that. good deal.
