Replacing a HD in a mirrored software raid

Discussion in 'Server Operation' started by Ovidiu, May 12, 2015.

  1. Ovidiu

    Ovidiu Active Member

    I'm running a perfect server Debian 7 setup according to the Perfect Server howto and have noticed an impending HD failure. See my initial thread too: https://www.howtoforge.com/community/threads/restoring-a-complete-server-with-duply-duplicity.70027/

    My ISP, STRATO, now has offered to replace the failed hard drive out of courtesy so I was googling how to replace a failed hard drive in a mirrored software raid and found this how-to: https://www.howtoforge.com/replacing_hard_disks_in_a_raid1_array

    But it looks like my array isn't damaged even though smartctl reports it will fail very, very soon:

    Code:
    SMART Attributes Data Structure revision number: 16
    Vendor Specific SMART Attributes with Thresholds:
    ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE
    
    5 Reallocated_Sector_Ct PO--CK 001 001 005 NOW 796

    Code:
    cat /proc/mdstat
    Personalities : [raid1]
    md1 : active raid1 sdb3[0] sda3[1]
    1948392384 blocks [2/2] [UU]
    [==>..................] resync = 10.3% (200765440/1948392384) finish=7547.6min speed=3858K/sec
    
    md0 : active raid1 sdb1[0] sda1[1]
    1023936 blocks [2/2] [UU]
    
    unused devices: <none>
    So I was wondering what would the procedure be if the HD simply failed and was replaced before I had the chance to first removing the faulty drive via:
    Code:
    mdadm --manage /dev/md1 --fail /dev/sda3
    and
    mdadm --manage /dev/md0 --fail /dev/sda1
     
  2. till

    till Super Moderator Staff Member ISPConfig Developer

    The raid seems to get resynced at the moment. so if the harddisk has been replaced already, then just wit ntil the resync is finished.
     
  3. Ovidiu

    Ovidiu Active Member

    nope, that was what it looked like before being replaced.
    I've replaced it now, followed the tutorial and it is busy resyncing.

    Will psot back here after its finished.
     
    Last edited: May 12, 2015
  4. sjau

    sjau Local Meanie Moderator

    did it work meanwhile?
     
  5. Ovidiu

    Ovidiu Active Member

    jepp, all good. Strato said they'd email me but no such thing. They turned the server off and replaced the HD. I only noticed when my monitor tool complained that my sites were offline. All I had to do from the tutorial I linked to was copy the partition layout and assign the new HD to the 2 raids, then all went automatically and is good now!
     
  6. sjau

    sjau Local Meanie Moderator

    also make sure that boot loader is installed on the new hd :)
     
  7. Ovidiu

    Ovidiu Active Member

    @sjau: to be honest, I am way too scared of doing that. how could I check if it is actually necessary?
    sda was broken and they pulled it. when they replaced it and rebooted apparently the system had automatically set sba to sda? and the system booted just fine? a bit confused here...
    or after reading the comments on the howto, maybe the technicians unplugged sdb and placed it into the other slot so I could boot off it?

    do I really only need to do a "grub-install /dev/sdb" ? Running latest Debian 7 here.
     
  8. sjau

    sjau Local Meanie Moderator

    well, not sure what they did... I maintain my raids myself. So without a boot loader installed on the other raid drives then they won't boot if the "main" one fails.
     
  9. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    With grub you can use "apt-get install --reinstall grub-pc" and select all /dev/sd?
     
  10. Ovidiu

    Ovidiu Active Member

    Hope you didn't get me wrong, the technicians definitely didn't have access to the system so the most they did was probably unplug the broken sda and plugged sdb into that same slot? Or does that not make any difference?

    Anyway, my question is: how could I check if it is actually necessary?

    @florian030 thanks for that advice, any way to check where the bootloader is currently installed?
     
  11. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    Strato just replace the failed hhd. If you raid is synced (cat /proc/mdstat) everything is ok. To find where grub is installed, you can read the first 512 bytes of the device (sda and sdb) with dd.
    If you are using grub2, just ran the command above.
     
    Ovidiu likes this.
  12. Ovidiu

    Ovidiu Active Member

    looks like I'm still on grub-legacy
     
  13. florian030

    florian030 ISPConfig Developer ISPConfig Developer

  14. Ovidiu

    Ovidiu Active Member

    hm, thanks, I'll google the rest. that link shows a GUI installation of grub into /dev/mdX
     

Share This Page