Our business has been configuring a new DNS Start of Authority server. It's a VM on Aventurin{e}, running Debian 12 and PHP 8.2.7. Our Apache version is 2.4.57, but we use another server for our webservers. It also runs our company IRC daemon, InspIRCd, but that doesn't interact with ISPConfig at all.The DNS is functioning, however this morning we found that we were unable to log in to the ISPConfig web console. The login page appeared, but upon entering the username and password, it displayed a white screen. After rebooting the server, the issue persisted until we shut down the apache server. In the logs below, the most striking warning is the repeated "Database connection failed" messages relating to a PHP Fatal error. I saw another post describing a similar error from yesterday, but I'm making a new post because our situations are very different. I presume that this was caused by some sort of update, as many PHP files on the server show up as updated this morning, but we've been on the latest PHP version since this server's inception. Below are excerpts from logs I believe are relevant. ISPConfig's cron.log: Code: Fri Oct 6 12:55:02 EDT 2023 06.10.2023-16:55 - WARNING - There is already a lockfile set, but no process running with this pid (1461). Continuing. Fri Oct 6 12:55:12 EDT 2023 Database connection failed Database connection failed Database connection failed Database connection failed Database connection failed Zugriff auf Datenbankserver fehlgeschlagen! / Database server not accessible! PHP Fatal error: Uncaught Error: Call to a member function testConnection() on bool in /usr/local/ispconfig/server/server.php:64 Fri Oct 6 12:55:12 EDT 2023 Stack trace: Fri Oct 6 12:55:12 EDT 2023 #0 {main} Fri Oct 6 12:55:12 EDT 2023 thrown in /usr/local/ispconfig/server/server.php on line 64 Fri Oct 6 12:56:01 EDT 2023 06.10.2023-16:56 - WARNING - There is already a lockfile set, but no process running with this pid (1487). Continuing. Fri Oct 6 12:56:11 EDT 2023 Database connection failed Database connection failed Database connection failed Database connection failed Database connection failed Zugriff auf Datenbankserver fehlgeschlagen! / Database server not accessible! PHP Fatal error: Uncaught Error: Call to a member function testConnection() on bool in /usr/local/ispconfig/server/server.php:64 Fri Oct 6 12:56:11 EDT 2023 Stack trace: Fri Oct 6 12:56:11 EDT 2023 #0 {main} Fri Oct 6 12:56:11 EDT 2023 thrown in /usr/local/ispconfig/server/server.php on line 64 Fri Oct 6 12:57:01 EDT 2023 PHP Warning: Undefined variable $state_out in /usr/local/ispconfig/server/lib/classes/plugins.inc.php on line 173 Fri Oct 6 12:57:01 EDT 2023 finished server.php. Apache2's error.log: Code: [Fri Oct 06 12:48:49.923527 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: PHP Fatal error: Uncaught TypeError: array_filter(): Argument #1 ($array) must be of type array, bool given in /usr/local/ispconfig/interface/web/login/index.php:199, referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923573 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: Stack trace:, referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923584 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: #0 /usr/local/ispconfig/interface/web/login/index.php(199): array_filter(), referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923593 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: #1 /usr/local/ispconfig/interface/web/login/index.php(81): is_admin_ip_whitelisted(), referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923603 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: #2 /usr/local/ispconfig/interface/web/login/index.php(459): process_login_request(), referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923612 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: #3 {main}, referer: https://<server IP>:8080/login/ [Fri Oct 06 12:48:49.923620 2023] [fcgid:warn] [pid 908:tid 139788241766080] [remote ***.***.***.***:57438] mod_fcgid: stderr: thrown in /usr/local/ispconfig/interface/web/login/index.php on line 199, referer: https://<server IP>:8080/login/ [Fri Oct 06 12:58:55.188441 2023] [mpm_event:notice] [pid 627:tid 139788541822848] AH00492: caught SIGWINCH, shutting down gracefully Test script: Code: ##### SERVER ##### IP-address (as per hostname): ***.***.***.*** [WARN] could not determine server's ip address by ifconfig [INFO] OS version is Debian GNU/Linux 12 (bookworm) [INFO] uptime: 13:00:25 up 14 min, 2 users, load average: 0.00, 0.03, 0.05 [INFO] memory: total used free shared buff/cache available Mem: 16Gi 1.5Gi 13Gi 16Mi 617Mi 14Gi Swap: 2.0Gi 0B 2.0Gi [INFO] systemd failed services status: UNIT LOAD ACTIVE SUB DESCRIPTION ● ifupdown-pre.service loaded failed failed Helper to synchronize boot up for ifupdown ● systemd-networkd-wait-online.service loaded failed failed Wait for Network to be Configured LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 2 loaded units listed. [INFO] ISPConfig is installed. ##### ISPCONFIG ##### ISPConfig version is 3.2.11 ##### VERSION CHECK ##### [INFO] php (cli) version is 8.2.7 [INFO] php-cgi (used for cgi php in default vhost!) is version 8.2.7 ##### PORT CHECK ##### [WARN] Port 8081 (ISPConfig Apps) seems NOT to be listening [WARN] Port 143 (IMAP server) seems NOT to be listening [WARN] Port 993 (IMAP server SSL) seems NOT to be listening [WARN] Port 110 (POP3 server) seems NOT to be listening [WARN] Port 995 (POP3 server SSL) seems NOT to be listening [WARN] Port 465 (SMTP server SSL) seems NOT to be listening [WARN] Port 21 (FTP server) seems NOT to be listening ##### MAIL SERVER CHECK ##### [WARN] I found no "submission" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer port 587 for smtp connections you have to enable this. [WARN] I found no "smtps" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer SSL for smtp (not TLS) connections you have to enable this. ##### RUNNING SERVER PROCESSES ##### [INFO] I found the following web server(s): Apache 2 (PID 1723) [INFO] I found the following mail server(s): Postfix (PID 901) [WARN] I could not determine which pop3 server is running. [WARN] I could not determine which imap server is running. [WARN] I could not determine which ftp server is running. ##### LISTENING PORTS ##### (only () Local (Address) [anywhere]:3306 (413/mariadbd) ***.***.***.***:6667 (1127/inspircd) [localhost]:11211 (300/memcached) [anywhere]:111 (1/init) [anywhere]:8080 (1723/apache2) [anywhere]:80 (1723/apache2) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) ***.***.***.***:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [localhost]:53 (1079/named) [anywhere]:22 (346/sshd:) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:953 (1079/named) [localhost]:25 (901/master) [anywhere]:443 (1723/apache2) [anywhere]:2812 (483/monit) *:*:*:*::*:3306 (413/mariadbd) [localhost]11 (1/init) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:53 (1079/named) *:*:*:*::*:22 (346/sshd:) *:*:*:*::*:25 (901/master) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:953 (1079/named) *:*:*:*::*:2812 (483/monit) ##### IPTABLES ##### Chain INPUT (policy ACCEPT) target prot opt source destination f2b-sshd 6 -- [anywhere]/0 [anywhere]/0 multiport dports 22 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain f2b-sshd (1 references) target prot opt source destination REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable REJECT 0 -- ***.***.***.*** [anywhere]/0 reject-with icmp-port-unreachable RETURN 0 -- [anywhere]/0 [anywhere]/0 ##### LET'S ENCRYPT ##### acme.sh is installed in /root/.acme.sh/acme.sh Please let us know if you have any information on this issue. Thank you.
Check that you can log in to mysql/mariadb with the details found in the file /usr/local/ispconfigInterface/lib/config.inc.php. You can e.g. test that with the mysql command: mysql -h localhost -u ispconfig -p dbispconfig and enter the password from the config file when requested. if this works, test also: mysql -h 127.0.0.1 -u ispconfig -p dbispconfig
It's a single virtual machine. There are other virtual machines running on the host but none of them have any relation to this one.
Update: It looks like we had incorrect permissions on our admin IP whitelist file. A trivial error in hindsight, but a good one to keep in mind for those who encounter such an error in the future.
