My backup server become unresponsive when running simple commands, like viewing log files or simple timedatectl command. My syslog is full of the same error. Code: Dec 14 07:20:02 bckserver2 kernel: [2918103.848994] ata3.00: failed command: WRITE DMA EXT Dec 14 07:20:02 bckserver2 kernel: [2918103.849934] ata3.00: cmd 35/00:00:00:60:14/00:08:23:00:00/e0 tag 0 dma 1048576 out Dec 14 07:20:02 bckserver2 kernel: [2918103.849934] res 40/00:00:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout) Dec 14 07:20:02 bckserver2 kernel: [2918103.851821] ata3.00: status: { DRDY } Dec 14 07:20:02 bckserver2 kernel: [2918103.852765] ata3: soft resetting link Dec 14 07:20:03 bckserver2 kernel: [2918104.036230] ata3.00: configured for UDMA/33 Dec 14 07:20:03 bckserver2 kernel: [2918104.036237] ata3: EH complete Dec 14 07:20:34 bckserver2 kernel: [2918134.888009] ata3: lost interrupt (Status 0x50) Dec 14 07:20:34 bckserver2 kernel: [2918134.888018] ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen I use the server only as mirror off web and run rsync and mysql dump commands once per day. I'm running Ubuntu 14.04.5 lts server. Does any one know what the problem could be or a solution?
Can't becuse when I try to apt-get install smartmodtools it throws errors and mounts the fs as read-only. And on reboot I get "Errors were found while checking the disk drive for /.".
I guess your harddisks are dead or nearly as dead. if it's just a backup server, replace the harddisk and reinstall it.
With gparded live I run smartctl -a /dav/sda and got "Vendor Specific SMARTAttribute with Thresholds:" 3 Pre-fail other Old_age, "ATA errors count: 2" and 3 errors on disk power-on. So disk is nearly dead like you said?
This is the output, but the disk is dead. I run badblocks -svn /dev/sda and it's on 0.23% of the first test and there are 19/0/64 errors allready! Code: SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0027 187 122 021 Pre-fail Always - 3641 4 Start_Stop_Count 0x0032 099 099 000 Old_age Always - 1542 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 093 093 000 Old_age Always - 5532 10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 1520 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 631 193 Load_Cycle_Count 0x0032 095 095 000 Old_age Always - 317594 194 Temperature_Celsius 0x0022 125 097 000 Old_age Always - 22 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 68 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 76 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 2 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 44 SMART Error Log Version: 1 ATA Error Count: 2 CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 2 occurred at disk power-on lifetime: 5529 hours (230 days + 9 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 70 40 d0 07 e0 Error: UNC 112 sectors at LBA = 0x0007d040 = 512064 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 70 18 d0 07 e0 08 00:07:43.109 READ DMA c8 00 08 00 10 00 e0 08 00:07:43.088 READ DMA c8 00 88 80 dc 31 e0 08 00:07:43.064 READ DMA Error 1 occurred at disk power-on lifetime: 5529 hours (230 days + 9 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 40 d0 07 e0 Error: UNC 8 sectors at LBA = 0x0007d040 = 512064 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 40 d0 07 e0 08 00:05:54.863 READ DMA SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. I have a second disk from the second backup server that I crashed and I checked. The badblocks -svw /dev/sda run for 3 days and 0 errors. But can you look at the smartctl output for this one. Code: SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 080 063 044 Pre-fail Always - 106460754 3 Spin_Up_Time 0x0003 099 094 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 216 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 075 060 030 Pre-fail Always - 108237076237 9 Power_On_Hours 0x0032 050 050 000 Old_age Always - 44164 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 1 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 168 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 099 099 000 Old_age Always - 1 188 Command_Timeout 0x0032 100 099 000 Old_age Always - 2 189 High_Fly_Writes 0x003a 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0022 065 055 045 Old_age Always - 35 (Min/Max 30/37) 194 Temperature_Celsius 0x0022 035 045 000 Old_age Always - 35 (0 20 0 0 0) 195 Hardware_ECC_Recovered 0x001a 053 022 000 Old_age Always - 106460754 196 Reallocated_Event_Count 0x0033 100 100 036 Pre-fail Always - 0 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 SMART Error Log Version: 1 ATA Error Count: 1 CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 1 occurred at disk power-on lifetime: 38807 hours (1616 days + 23 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 00 68 f1 62 00 Error: UNC at LBA = 0x0062f168 = 6484328 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- 25 00 00 c8 ef 62 e0 00 37d+23:01:08.573 READ DMA EXT 25 00 00 c8 ee 62 e0 00 37d+23:01:08.464 READ DMA EXT 25 00 00 c8 ed 62 e0 00 37d+23:01:08.444 READ DMA EXT 25 00 00 38 a9 7c e3 00 37d+23:01:08.432 READ DMA EXT 35 00 08 d0 ed 5e e0 00 37d+23:01:08.431 WRITE DMA EXT SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 44097 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. On this disk when I try to install ubuntu 14.04.4 LTS server I get timeouts and the HP ProLiant ML310 G5 won't restart or reboot becouse of the timeouts.
