MySQL can't connect after system udpate

Discussion in 'Installation/Configuration' started by schwim, Apr 7, 2022.

  1. schwim

    schwim Member HowtoForge Supporter

    Hello there!
    I performed a system update with the intention of doing an ISPC update afterward. After the system update, however, I'm unable to perform the ISPC update due to an error:
    In another shell, I tried to connect to mysql but received the following error:
    I'm unsure how to proceed at this point. Any help on what to do next would be helpful. Thanks for your time!
     
  2. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    What is the output of
    Code:
    systemctl status mariadb
    ?
     
  3. schwim

    schwim Member HowtoForge Supporter

     
  4. Th0m

    Th0m ISPConfig Developer Staff Member ISPConfig Developer

    Try
    Code:
    systemctl restart mariadb
    What happens?

    If it still fails, check log: /var/log/mysql/error.log
     
  5. schwim

    schwim Member HowtoForge Supporter

    error.log:
    journalctl -xe:
     
  6. schwim

    schwim Member HowtoForge Supporter

    A lot of my Googling finds results regarding corrupt log files or database permissions in this directory so I thought I'd post an image of it in case there's something telling in the files I'm afraid to backup the directory and run a reinitialization because I think that means I'll lose all the mysql users, further breaking the ISPC install:

    [​IMG]
     
    Last edited: Apr 8, 2022
  7. till

    till Super Moderator Staff Member ISPConfig Developer

    Set a recovery mode in MariaDB and then start it in recovery mode. Start using the lowest level and use higher levels if needed:

    https://mariadb.com/kb/en/innodb-recovery-modes/

    and before you try these attempts, make a copy of /var/lib/mysql folder as a backup.
     
  8. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    What kind of update? What OS is this? If you upgraded to next release of the OS, you must follow ISPConfig Perfect server manual for that new OS to get needed packages installed and configurations fixed.
     
  9. schwim

    schwim Member HowtoForge Supporter

    I updated Debian 10 via apt-get upgrade and during this update, changed from STABLE to OLD_STABLE. Although I didn't change versions, does changing to old stable require me to make changes to get ISPC repos or settings?
     
  10. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    If you did apt-get upgrade with STABLE as release in apt sources file after Debian 11 was released, you have packages from Debian 11 installed on your system. So the system is something between 10 and 11, and may work badly and my guess is ISPConfig is confused.
    Instead of stable and old_stable, use the release code names. They do not change over time and your system does not break. So now stable is bullseye, and Debian 10 is still buster.
     
  11. schwim

    schwim Member HowtoForge Supporter

    From the OS:
    This is my sources.list
    Everything seems to still be set at Buster. Am I missing some obvious information or do I need to look in other locations to see what I need to change?
     
  12. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    You wrote:
    What exactly do you mean by that?
     
  13. schwim

    schwim Member HowtoForge Supporter

    From my understanding, it is something you must agree to to remain on 10 and not be moved to 10.5 . I got the error when trying to upgrade and my Googling led me to pages that showed that you had to explicitely agree to remain on your release before it would allow you to perform an upgrade. This was the page I found that showed me how to agree to not moving to the next release:

    https://forum.proxmox.com/threads/a...s-suite-value-from-stable-to-oldstable.94726/
     
  14. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    That thread is about Proxmox, not Debian GNU/Linux.
    If you all the time had buster in your apt sources.list file, all is good.
    Is this the first time you did apt upgrade since release of Debian 11 Bullseye?
     
  15. schwim

    schwim Member HowtoForge Supporter

    Yes, I tend to shy away from upgrades on the production machines for reasons exactly like this. This machine has been down for a day now after an updgrade.
     
  16. till

    till Super Moderator Staff Member ISPConfig Developer

    According to the errors you posted, there is no issue with an upgrade. You have an issue with corrupted MariaDb index and or log files. Such things can happen for many reasons and it is not an issue specific to updating software. See my post above on how to recover or use google and search for mariadb innodb recovery.
     
  17. schwim

    schwim Member HowtoForge Supporter

    I've been working on this since you've posted and I'm still unable to work through getting it running. I think I'm doing it correctly but the service continues to fail to start:
    Currently in /etc/my.cnf
     
    Last edited: Apr 8, 2022
  18. schwim

    schwim Member HowtoForge Supporter

    Ok, mysql starts at recovery level 6. However, I'm unable to do anything when it does start due to lack of permissions:
    What should I do at this point? Would I just remove each db from the directory until MySQL will start to find the problem database?

    [​IMG]
     
    Last edited: Apr 8, 2022
  19. Jesse Norell

    Jesse Norell Well-Known Member Staff Member Howtoforge Staff

    Add '-p' to prompt for the mysql root password (which is found in /usr/local/ispconfig/server/lib/mysql_clientdb.conf if needed).
     
  20. schwim

    schwim Member HowtoForge Supporter

    Thank you for that.

    It seems all of my dbs are ok. Now I'm completely lost:

    https://pastebin.com/kc109XB2
     

Share This Page