500 - Internal Server Error

I am running ispconfig with Nginx on Debian 9.1.3.
I had a crash and had to run fsck to via KVM to get the OS back and am still getting a 500 Internal server error on webpages and can't access IspConfig.

cat /var/log/nginx/error.log
.
.
.
2020/11/01 17:37:36 [error] 1284#1284: *1309 FastCGI sent in stderr: "PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 85
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 85
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP message: PHP Warning: mysqli_real_connect(): (HY000/2002): No such file or directory in /usr/local/ispconfig/interface/lib/classes/db_mysql.inc.php on line 91
PHP messa
2020/11/01 17:48:05 [error] 1287#1287: *1514 directory index of "/var/www/apps/" is forbidden, client: 185.172.111.197, server: _, request: "HEAD / HTTP/1.0"

I am not sure how to recover from this crash. Do I reinstall ISP config or??
This is not the first time this has happened on this server.
https://www.howtoforge.com/community/threads/error-500-internal-server-error.83728/
The server load is typically quite low.

Thanks for your help

 

What kind of crash? What is the filesystem on the affected partition?
Forcing ISPConfig update might write the ISPConfig files back on your disk. That might help. You wrote

but that can not be your Debian version. Do you know what ISPConfig version is on that disk?

 

Thanks for the reply.

I guess crashes can be a little vague so all I can do its explain what happened.

I was having problems with Google SEO and canonical domain names so I started playing around with alias domain names. I started seeing problems with the DNS where many different sites pointed to a went to a single site. Eventually I could not access any pages or ISP config and would get a 500 error. I was able to ssh into the system and perform an update but after a reboot or other I ended up losing the ability to ssh into the server. I had the host set up a KVM and I was able to fsck /dev/mapper/pc--vg-root (ext4) and am now able to ssh into the server but still get 500 errors.

The Debian version is 9.13

I am not sure but I believe that I am still running IspConfig 3.1

 

I attempted to do an update form Ispconfig3.1 to 3.2 and it failed.

Unable to connect to mysql server No such file or directory
MySQL root password [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]:

 

Did you have a reason for doing that?
Did you follow the instructions for updating ISPConfig 3.1 to 3.2?

 

I did not have a better solution and ...
"Forcing ISPConfig update might write the ISPConfig files back on your disk. That might help."
Yes I followed the instructions and updated the proper applications first.

 

Thanks for the reply...good call ... yes this appears to be a problem
root@mr1:~# service mysqld status

● mariadb.service - MariaDB 10.1.47 database server
Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/mysql.service.d
└─limits.conf
Active: activating (auto-restart) (Result: signal) since Mon 2020-11-02 14:21:48 CST; 2s ago
Docs: man:mysqld(8)
https://mariadb.com/kb/en/library/systemd/
Process: 2145 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=killed, signal=ABRT)
Process: 2067 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= || VAR=`cd /usr/bin/..; /usr/bin/galera_recovery`; [ $? -eq 0 ] && sys
Process: 2063 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
Process: 2060 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
Main PID: 2145 (code=killed, signal=ABRT)
Status: "Starting Innodb crash recovery"

Nov 02 14:21:48 mr1 systemd[1]: mariadb.service: Unit entered failed state.
Nov 02 14:21:48 mr1 systemd[1]: mariadb.service: Failed with result 'signal'.

 

Any repair suggestions? This had many word press sites. There are many databases and tables.

 

I am not sure what the best approach is.

root@mr1:~# mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")

 

root@mr1:/etc/mysql# mysqlcheck --auto-repair -o --all-databases -uroot -pXXXXXXXXXXXXXXXXXXXXXX
mysqlcheck: Got error: 2002: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory") when trying to connect

**** Edit *****
mariadb is installed
root@mr1:~# dpkg -s mariadb-server
Package: mariadb-server
Status: install ok unpacked

Any suggestions comments or other?

 

Do I need to add/generate a mysqld.sock ???
What would be the best approach here??

 

Thanks for the reply.
I think that this is my situation.
https://www.howtoforge.com/community/threads/maria-db-will-not-start.82470/

The question here is what is my best course of action.

 

I made a copy of the directory and used the procedure in the link provided. I had reasonable results as I was able to get ISP Config working and some of the websites. However I was not able to recover WordPress sites without a backup. The databases existed but the tables were empty or had issues.

I attempted to run the procedure again with my backups and completely messed the database and ISP config.

Code:

service mysqld start
Job for mariadb.service failed because the control process exited with error code.
See "systemctl status mariadb.service" and "journalctl -xe" for details.

Unless somebody has a good Idea on how to recover I feel that this one is toast.

I would like to recover a few tar files but can not get them with FileZilla

Code:

Command:   PASS *********************
Response:   530 Login authentication failed
Error:   Critical error: Could not connect to server

Any suggestions as how to get ftp working?

Thanks