Hi Had to reboot my server as the mysql service just randomly stopped working. I was then greeted with a disk failure is imminent message which i've managed to recover from but the mysql service still doesnt come up. So far I've tried a yum update mysqld and that updated some of the modules but the service still doesnt come up. When I enter service mysqld start I just get a failed message and no more information. I need to access the mysql databases to back everything up before the HDD dies. Any ideas on what the problem is? Would a yum remove mysqld and then reinstall work or would I loose all my databases?
First you should do a harddisk backup of your mysql databases by creating a tar.gz file of the mysql database folder. In case of a harddisk failure you should be able to use that to recover the databases: tar pcfz mysql_databases.tar.gz /var/lib/mysql Then you should check if there are any error messages in the syslog or messages log in /var/log/ when you restart mysql You should also check with "df -h" if none of the harddisk partitions are full. When mysql is working again, then you should do a normal mysql backup with e.g mysqldump.
thanks for the pointer till I've done the back up as suggested I'm only using 17% of the disk so I have plenty of space The error log contains this: Code: 110706 10:47:30 mysqld started InnoDB: Error: tried to read 65536 bytes at offset 0 4124160. InnoDB: Was only able to read 512. InnoDB: Fatal error: cannot read from file. OS error number 17. 110706 10:47:48InnoDB: Assertion failure in thread 3087247152 in file os0file.c line 2211 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html InnoDB: about forcing recovery. 110706 10:47:48 - mysqld got signal 11; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=0 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217599 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xbff53f88, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x818b124 0x842e63e 0x83ff548 0x83dd267 0x83e7030 0x83e75d2 0x83482c7 0x8254e76 0x8244b1f 0x818c10d 0x818e3ed 0x8275d6 0x80fc111 New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it The manual page at http://www.mysql.com/doc/en/Crashing.html contains information that should help you find out what is causing the crash. 110706 10:47:48 mysqld ended 110706 10:54:36 mysqld started 110706 10:54:36 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295 110706 10:54:36 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295 InnoDB: Error: tried to read 65536 bytes at offset 0 4124160. InnoDB: Was only able to read 512. InnoDB: Fatal error: cannot read from file. OS error number 17. 110706 10:54:55InnoDB: Assertion failure in thread 3086681904 in file os0file.c line 2211 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html InnoDB: about forcing recovery. 110706 10:54:55 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=0 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217599 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xbfdc95f8, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x818f189 0x843940e 0x840a218 0x83e7f07 0x83f1d00 0x83f22a2 0x8352db7 0x825e6f2 0x824e4af 0x819018d 0x819248d 0x8275d6 0x80fd501 New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 110706 10:54:55 mysqld ended 110706 11:04:50 mysqld started 110706 11:04:50 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295 110706 11:04:50 [Warning] option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295 InnoDB: Error: tried to read 65536 bytes at offset 0 4124160. InnoDB: Was only able to read 512. InnoDB: Fatal error: cannot read from file. OS error number 17. 110706 11:05:08InnoDB: Assertion failure in thread 3087075120 in file os0file.c line 2211 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.0/en/forcing-recovery.html InnoDB: about forcing recovery. 110706 11:05:08 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=0 read_buffer_size=131072 max_used_connections=0 max_connections=100 threads_connected=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 217599 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd=(nil) Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... Cannot determine thread, fp=0xbfe29e58, backtrace may not be correct. Stack range sanity check OK, backtrace follows: 0x818f189 0x843940e 0x840a218 0x83e7f07 0x83f1d00 0x83f22a2 0x8352db7 0x825e6f2 0x824e4af 0x819018d 0x819248d 0x8275d6 0x80fd501 New value of fp=(nil) failed sanity check, terminating stack trace! Please read http://dev.mysql.com/doc/mysql/en/using-stack-trace.html and follow instructions on how to resolve the stack trace. Resolved stack trace is much more helpful in diagnosing the problem, so please do resolve it The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 110706 11:05:08 mysqld ended
