Hello, I am experiencing some issues which I am hoping I can get some help with. I noticed today that I am unable to modify any website settings and the job just sits in the queue and the following is just output over and over in the /var/log/ispconfig/cron.log Code: Mon Jan 13 02:28:01 GMT 2025 13.01.2025-02:28 - WARNING - There is already a lockfile set, but no process running with this pid (12184). Continuing. Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [server:184] - Found 1 changes, starting update process. Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [plugins.inc:118] - Calling function 'ssl' from plugin 'apache2_plugin' raised by event 'web_domain_update'. Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'apache2_plugin' raised by event 'web_domain_update'. Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0 Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0 Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: df -T '/var/www/clients/client9/web91'|awk 'END{print $2,$NF}' - return code: 0 Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: which 'setquota' 2> /dev/null - return code: 0 Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -u 'web91' '0' '0' 0 0 -a &> /dev/null - return code: 0 Mon Jan 13 02:28:02 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -T -u 'web91' 604800 604800 -a &> /dev/null - return code: 0 Mon Jan 13 02:28:03 GMT 2025 13.01.2025-03:28 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0 I believe the last change I made to the server was an ispconfig update during November but I never tested the website configuration changes and the job queue after this so I can't say if it's been broken for longer. I did install additional PHP versions 7.2 & 7.4 but this was done quite a while ago and I can see that I have assigned these PHP versions to some of my websites so it must have been working normally. I have tested updating DNS/FTP/Mail etc and these all work fine is just any changes made to websites. I have tried creating a new test website and the directories in /var/www/clients/ get created but the the vhosts file does not. I have checked the permissions and these are all good. I can see the modification time stamp on ispconfig.conf/.vhost is today but I am not sure if this is just because I ran the update script with --force to see if that would resolve anything. Either way it shows that the directory/files can be written to. Below is the output from the test script and df -h for now I have just changed the incremented value in the updated column of the server table, thanks. Code: ##### SERVER ##### IP-address (as per hostname): ***.***.***.*** [WARN] could not determine server's ip address by ifconfig [INFO] OS version is "CentOS Linux release 7.9.2009 (Core)" [INFO] uptime: 02:21:07 up 2:29, 4 users, load average: 1.77, 1.79, 1.44 [INFO] memory: total used free shared buff/cache available Mem: 15G 5.8G 162M 71M 9.6G 9.4G Swap: 510M 0B 510M [INFO] systemd failed services status: UNIT LOAD ACTIVE SUB DESCRIPTION ● ipmi.service loaded failed failed IPMI Driver ● kdump.service loaded failed failed Crash recovery kernel arming LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 2 loaded units listed. Pass --all to see loaded but inactive units, too. To show all installed unit files use 'systemctl list-unit-files'. [INFO] ISPConfig is installed. ##### ISPCONFIG ##### ISPConfig version is 3.2.12p1 ##### VERSION CHECK ##### [INFO] php (cli) version is 5.6.40 [INFO] php-cgi (used for cgi php in default vhost!) is version 5.6.40 [WARN] You are using an outdated php version. ##### PORT CHECK ##### [WARN] Port 465 (SMTP server SSL) seems NOT to be listening ##### MAIL SERVER CHECK ##### [WARN] I found no "submission" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer port 587 for smtp connections you have to enable this. [WARN] I found no "smtps" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer SSL for smtp (not TLS) connections you have to enable this. ##### RUNNING SERVER PROCESSES ##### [INFO] I found the following web server(s): Unknown process (httpd) (PID 6915) [INFO] I found the following mail server(s): Unknown process (smtpd) (PID 8780) [INFO] I found the following pop3 server(s): Dovecot (PID 15153) [INFO] I found the following imap server(s): Unknown process (dovecot/imap-l) (PID 2786) [INFO] I found the following ftp server(s): PureFTP (PID 15031) ##### LISTENING PORTS ##### (only () Local (Address) [localhost]:9016 (15352/php-fpm:) [anywhere]:25 (8780/smtpd) [localhost]:953 (22117/named) [localhost]:9017 (15352/php-fpm:) [localhost]:9056 (1087/php-fpm:) [anywhere]:993 (2786/dovecot/imap-l) [anywhere]:995 (15153/dovecot) [localhost]:10024 (7089/amavisd) [localhost]:9000 (6866/php-fpm:) [localhost]:10025 (26484/master) [localhost]:10026 (7089/amavisd) [localhost]:10027 (26484/master) [localhost]:9100 (1632/noderig) [anywhere]:110 (15153/dovecot) [localhost]:9007 (15327/php-fpm:) [anywhere]:143 (2786/dovecot/imap-l) [anywhere]:111 (762/rpcbind) ***.***.***.***:53 (22117/named) [localhost]:53 (22117/named) [anywhere]:21 (15031/pure-ftpd) [anywhere]:22 (1065/sshd) [localhost]:8120 (4966/java) *:*:*:*::*:953 (22117/named) *:*:*:*::*:8218 (5148/java) *:*:*:*::*:8219 (2698/java) *:*:*:*::*:443 (6915/httpd) *:*:*:*::*:8220 (4966/java) [localhost]:1213 (5199/java) *:*:*:*::*:8225 (2698/java) *:*:*:*::*:993 (2786/dovecot/imap-l) *:*:*:*::*:995 (15153/dovecot) *:*:*:*::*:2213 (5199/java) *:*:*:*::*:10024 (7089/amavisd) *:*:*:*::*:10026 (7089/amavisd) *:*:*:*::*:3306 (14924/mysqld) [localhost]:8108 (4163/java) [localhost]10 (15153/dovecot) [localhost]43 (2786/dovecot/imap-l) [localhost]11 (762/rpcbind) *:*:*:*::*:8080 (6915/httpd) *:*:*:*::*:80 (6915/httpd) [localhost]:8112 (5416/java) *:*:*:*::*:8208 (4163/java) *:*:*:*::*:8081 (6915/httpd) *:*:*:*::*:8210 (4163/java) *:*:*:*::*:8212 (5416/java) *:*:*:*::*:53 (22117/named) *:*:*:*::*:21 (15031/pure-ftpd) *:*:*:*::*:8213 (5416/java) [localhost]:8118 (5148/java) *:*:*:*::*:22 (1065/sshd) [localhost]:8119 (2698/java) ##### IPTABLES ##### Chain INPUT (policy ACCEPT) target prot opt source destination f2b-postfix-sasl tcp -- [anywhere]/0 [anywhere]/0 multiport dports 25,465,587 f2b-sshd tcp -- [anywhere]/0 [anywhere]/0 tcp dpt:22 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain f2b-postfix-sasl (1 references) target prot opt source destination REJECT all -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable RETURN all -- [anywhere]/0 [anywhere]/0 Chain f2b-sshd (1 references) target prot opt source destination REJECT all -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT all -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable RETURN all -- [anywhere]/0 [anywhere]/0 ##### LET'S ENCRYPT ##### Certbot is installed in /usr/bin/letsencrypt Code: df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 7.8G 0 7.8G 0% /dev tmpfs 7.8G 0 7.8G 0% /dev/shm tmpfs 7.8G 19M 7.8G 1% /run tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup /dev/sda1 1.8T 893G 848G 52% / tmpfs 1.6G 0 1.6G 0% /run/user/0 tmpfs 1.6G 0 1.6G 0% /run/user/1001 tmpfs 1.6G 0 1.6G 0% /run/user/5007
Please post the full output that you get on the shell when you run server.sh: https://www.faqforge.com/linux/debugging-ispconfig-3-server-actions-in-case-of-a-failure/
Thanks for your reply, here is the the output. Code: 13.01.2025-11:04 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. 13.01.2025-11:04 - DEBUG [server:184] - Found 1 changes, starting update process. 13.01.2025-11:04 - DEBUG [plugins.inc:118] - Calling function 'ssl' from plugin 'apache2_plugin' raised by event 'web_domain_update'. 13.01.2025-11:04 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'apache2_plugin' raised by event 'web_domain_update'. 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: df -T '/var/www/clients/client9/web91'|awk 'END{print $2,$NF}' - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: which 'setquota' 2> /dev/null - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -u 'web91' '0' '0' 0 0 -a &> /dev/null - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -T -u 'web91' 604800 604800 -a &> /dev/null - return code: 0 13.01.2025-11:04 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client9/web91' - return code: 0
It looks incomplete; you must either have 'finished server.php.' as the last line or an PHP error message. Run server.sh again and post all lines from first to last one.
That's what's strange because that is the full output. Out of interest I tried deleting a random test site and that worked fine the problem seems to be just updating current settings or adding new ones. I do have newer versions of PHP available should I try running the server.php using a newer version? Code: [root@webserver ~]# php -v PHP 5.6.40 (cli) (built: Jun 5 2024 10:48:13) Copyright (c) 1997-2016 The PHP Group Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies [root@webserver ~]# /usr/local/ispconfig/server/server.sh 13.01.2025-11:36 - DEBUG [plugins.inc:155] - Calling function 'check_phpini_changes' from plugin 'webserver_plugin' raised by action 'server_plugins_loaded'. 13.01.2025-11:36 - DEBUG [server:184] - Found 1 changes, starting update process. 13.01.2025-11:36 - DEBUG [plugins.inc:118] - Calling function 'ssl' from plugin 'apache2_plugin' raised by event 'web_domain_update'. 13.01.2025-11:36 - DEBUG [plugins.inc:118] - Calling function 'update' from plugin 'apache2_plugin' raised by event 'web_domain_update'. 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client27/web28' - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client27/web28' - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: df -T '/var/www/clients/client27/web28'|awk 'END{print $2,$NF}' - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: which 'setquota' 2> /dev/null - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -u 'web28' '0' '0' 0 0 -a &> /dev/null - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: setquota -T -u 'web28' 604800 604800 -a &> /dev/null - return code: 0 13.01.2025-11:36 - DEBUG [system.inc:2436] - safe_exec cmd: chattr -i '/var/www/clients/client27/web28' - return code: 0
Really strange. But you get the shell prompt back, so it's not a hanging process? You can try that; just ensure it's not 8.4, as this version is not supported yet.
I get a shell prompt back so it's not a hanging process. I have tried running the server.php using PHP 7.2 but the output is the same sadly. I am so confused by this issue, there should be some error somewhere saying why it's not continued with the job but there is not. It run's stops and then runs again and stops at the same point.
Seems as if the PHP process dies unexpectedly then, as you should get an error message otherwise, either from ISPConfig or PHP or alternatively the finished message. The error from your first post: is also an indication for this issue as it means that the lock file created when server.sh starts has not been cleanly removed. But debugging this without errors is not easy at all, all we know is the last thing it is able to do is to set the quota, then it removes the attributes and fails. But as it removed the immutable attribute earlier already, removing the attribute is not a failure; it must happen right after that.
