recovery from failed update

Hi,
I updated my mail server today, after a long time for a reboot, i investigated it and i had a grub prompt up. i have tried to get the system to boot and it wont, either goes into busybox prompt if i select a previous kernel or if i choose latest it crashes.
So, i have it in recovery off a loaded cd disk, on lan with drive mounted. i have another identical raid server install debian as i type.
Can i just install needed binaries following setup of perfect debian server and then scp all the directories containing ispconfig files??
as it is a mail server, it only has /var/vmail/.... etc
And i need it up and running like 2 hours ago.. (well chief, you should have made time for the backup cloning of primary mail server you were planning for last 6 months) yes yes i know!!

please help obi wan

 

First find out what OS is running on that host.
You wrote

but please tell what kind of update? Just apt update or apt dist-upgrade and update of what, the OS, ISPConfig or both or something else?
Did you get errors or warnings during the update? What errors do you get now that booting fails? Try finding out what is wrong, then fix that.
There is this Tutorial, I would try this or other similar documents first to repair the boot:
https://www.howtoforge.com/tutorial/repair-linux-boot-with-grub-rescue/
Read also the comments, they usually have good info on the tutorial.
Assuming the update you did installed a working kernel, you should get back to a successful boot by reconfiguring grub to boot the right kernel with the approprieate settings. Or a previous kernel.

 

confimation - multi server setup, debian 9.12, ispconfig 3.1.15 - web, 2 x dns, 1 mail server - the mail server is the issue
All i did was

Code:

apt-get update

then

Code:

apt-get upgrade

, i rebooted the server and it came to stop at grub-rescue.
I have followed your link already, didn't fix my issue.

Code:

ls

- this lists (hd0,gpt2) as my ext partition
i try

Code:

set prefix=(hd0,gpt2)/boot/grub

Code:

set root=(hd0,gpt2)

Code:

insmod normal

- this gives me error: attempt to read or write outside of disk (hd0)
and this is odd..

Code:

ls (hd0,gpt2)/

- this lists all directories

Code:

ls (hd0,gpt2)/boot

- gives error attempt to read or write outside of disk (hd0)

So, i have another same dell R710 server 3 x 2TB SAS drives RAID 5, which i installed debian 9.12, all goes well until reboot and gives same error. i have tried again and again - same result.
I have done same install using debian 10 - also same result.
all attempts end in grub-rescue and cannot boot, i try and pointgrub using set command to kernel .

So,for now i have installed debian 9.12 on a singleHD machine (working) to just get the mail server running again, i want to compress the ispconfig 3 from old system mounted using rescue disk.. (its compressing /var/vmail now-
what directories in total do i need to move (compress) to reinstate my mail server..
/var/vmail/, /usr/local/ispconfig/, /etc/passwd/, /etc/group/, /var/lib/mysql and any other directories that ispconfig used to run

 

I would advise you to access and backup the server's contents (file, database, config etc) as there could be hdd failure or more, and only once that is done, then try to revive it.

 

So you have three same model machines that all fail in the same fails to boot way? This makes me think the new kernel or grub is at fault, or maybe driver for raid card or something. Have you checked debian bug reports? Or forums for dell R710 server?
Did you previously install some driver or kernel module from outside Debian on those hosts?

 

No, different dell raid server for web / control panel - that works.
But on Dell R710, that machine has 4 x 2TB all RAID 5 together - this is the mail server. This is the one that refuses to load. all files are there, just having errors when accessing /boot/..... that tells me

Code:

error attempt to read or write outside of disk (hd0)

it is also the same when i attempt a fresh install on spare server, finish install reboot then i get the same error.
Its all very odd.
Anyway, my main concern is to move the ispconfig and all the mail dirs to another pc..
can you please reply with the exact directories i can drop on to another standard PC thattemporarily will become mail server.

thanks

 

Thanks Till,
As i only have old server in recovery mode, i added

Code:

/var/lib/mysql/*

and compressed the databases then sent to replacement server. I have rebooted the system and mail is acepting people loggin in, but have issues.
I have tried ispconfig_update.sh it errors at the end with

Code:

Checking ISPConfig database .. OK
PHP Fatal error:  Uncaught Error: Call to undefined function mb_detect_encoding() in /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php:448
Stack trace:
#0 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php(166): db->escape('validate_passwo...')
#1 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php(231): db->_build_query_string('SELECT plugin_n...', Array)
#2 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php(270): db->_query('SELECT plugin_n...', Array)
#3 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php(319): db->query('SELECT plugin_n...', Array)
#4 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/update.lib.php(139): db->queryAllRecords('SELECT plugin_n...', Array)
#5 /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/update.php(315): updateDbAndIni()
#6 {main}
  thrown in /tmp/update_stable.sh.sA0nJZ1vyM/ispconfig3_install/install/lib/mysql.lib.php on line 448

seeing 1st line, google it and i get a result install

Code:

php-mbstring

Code:

The following NEW packages will be installed:
  php-mbstring php7.0-mbstring
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/470 kB of archives.
After this operation, 1,522 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
dpkg: unrecoverable fatal error, aborting:
 unknown group 'Debian-exim' in statoverride file
E: Sub-process /usr/bin/dpkg returned an error code (2)

An update / edit
look like a user in

Code:

/var/lib/dpkg/statoverride

i have backed up the file and install php-mbstring and it looks like it updating

 

new error

Code:

Jun 27 12:21:43 mail postfix/postdrop[438]: warning: mail_queue_enter: create file maildrop/816459.438: Permission denied

changed permissions

Code:

chown -R postfix:postfix /var/spool/postfix

not displaying permission error now.

Code:

postfix check
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/.
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/pid
postfix/postfix-script: warning: not owned by postfix: /var/lib/postfix/.
postfix/postfix-script: warning: not owned by postfix: /var/lib/postfix/./master.lock
postfix/postfix-script: warning: not owned by group postdrop: /var/spool/postfix/public
postfix/postfix-script: warning: not owned by group postdrop: /var/spool/postfix/maildrop
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/etc
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/etc/ssl
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/etc/ssl/certs
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/etc/ssl/certs/ca-certificates.crt
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/lib
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/lib/x86_64-linux-gnu
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/usr
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/usr/lib
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/usr/lib/sasl2
postfix/postfix-script: warning: not owned by root: /var/spool/postfix/usr/lib/zoneinfo

is this alright?
Fixed it..
Ok,

Code:

postfix check

reported incorrect permissions. checking my old system shows different permissions and some sub folders owned by postfix and postgrey with some sub folders with a variety of both, went though setting permissions and now sorted no more errors in /var/log/mail.log and the postfix mailcheck is clear.
sending emails again.
thank you all for the point in to right direction