Mysql would not start - MUCH FUN AND GAMES - AND SOLUTION!

Discussion in 'Tips/Tricks/Mods' started by craig baker, Mar 22, 2021.

  1. craig baker

    craig baker Member HowtoForge Supporter

    customer server lost power yesterday and they called me at 10am frantic!

    Sure enough websites down. much down because mariadb was not running! Could not get into ISPCONFIG, Nextcloud or anything!

    log contained:
    Ie NASTY. mariadb service could not run.
    now after much perusal I added a line in /etc/my.cnf
    port = 8881
    innodb_force_recovery=1

    note - the port = 8881 command is important to keep OTHER PROCESSES FROM DOING DB FUNCTIONS while we
    are recovering! remove it when done and delete the force_recovery line to get back to normal operation
    NOW systemctl start mariadb did NOT fail, and I was able to do:
    mysqlcheck -u rootuser -p --all-databases

    Note - when innodb_force_recovery=1 lets the service START - our problem is pretty much just corrupted database file!

    and one of my database had 2 corrupt tables. I exported the base with mysqldump and then dropped the 2 corrupt tables.
    all good? commented out the force_recovery and port commands and systemctl restart mariadb FAILED!!

    ARRGH TIL WHERE ARE YOU?????

    after calming down a bit --
    Now the gobbdegook on the mariadb/logfile referred to Innodb failure without naming a database. This turned out to mean
    the the ibfile(s) in /var/lib/mysql (adjust for your distribution here) were hosed.
    ibdata1 is a big file containing ALL innodb databases.
    I dumped all databases in recovery mode
    mysqldump -u rootuser -p -all-databases > ALLDATABASES.sql
    and it backed up all my databases successfully.
    Now rm ibdata1 and iblogfile*
    and go back to normal mode (comment innodb_force_recovery LEAVE port command to keep others from starting requests!)
    restart mariadb - SUCCESS!!!
    now we need to reimport all databases:

    mysql -u rootuser -p < ALLDATABASES.sql
    and we are back at a command prompt! successful.
    NOW comment out the port command restart mariadb and we have LIGHT

    all things work fine!
    hope others benefit from the entertainment!
    (note the NON THREATENING SUBJECT LINE)
    cdb.
     
    Last edited: Mar 22, 2021

Share This Page