Author Topic: Problems replacing failed hard drive TS-QVH12TL/R6  (Read 13383 times)

atlupo

  • Calf
  • *
  • Posts: 12
Problems replacing failed hard drive TS-QVH12TL/R6
« on: October 26, 2014, 05:24:34 pm »
A little help please with disk replacement procedure for TS-QVH12L/R6 (Firmware 1.22, RAID 5).

When I bought the NAS 3 years ago I also purchased a spare hard drive (exact same model number ST3000DM001) for the day when a drive might fail. Well that day has arrived – dreaded flashing red light and “HD4 Broken E30 Replace the DISK” error message.

So I put the new drive in and the darn box just sits there and does nothing – same flashing red light, same error messages. Selecting “Rediscover Disk” from admin webpage still gives disk error. Rebooting Terastation and waiting still gives disk error. I got the impression from a youtube video that this might take some time – I’ve now waited 1 hour and still nothing.

Standard procedure is to wait for message to start rebuilding array and hit “function” button but I never get the message to rebuild the array.
What am I doing wrong?

Do I really need to buy the replacement HD direct from Buffalo for twice the price I paid somewhere else?
Drive I bought came unformatted – do I need to format in some way to make it work with the NAS? Is that all Buffalo does to justify doubling the price?
Does what I describe mean something wrong in the box and not the hard drive (new and/or old)?

Any thoughts would be greatly appreciated.

Thanks very much.

Update: I was able to SSH into the NAS.  fdisk -l only shows the first 3 hard drives (/dev/sda, /dev/sdb, /dev/sdc). This behavior is with both old (failed?) HD as well as new HD plugged into the NAS. Where is /dev/sdd? Does this mean not a drive problem but rather some other problem with motherboard, SATA controller or cables/connectors in the NAS?
« Last Edit: October 27, 2014, 10:28:02 pm by atlupo »

joma90

  • VIP
  • *
  • Posts: 565
  • Life would be easy if we only had the source code!
    • Let me show you how to google
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #1 on: October 28, 2014, 08:52:10 am »
So i just confirming the new disk is in. If so, login to the UI of the unit and go to the settings where it shows the hdds/array, there will be a option in the array area to rebuild the unit.

Here is a guide on how to do it.
http://manual.buffalo.jp/buf-doc/35011719-02_EN.pdf
Units I own: TS-XL, TS-WXL, LS-WVL, TS-RXL,  TS5400, TS-RVHL,

I don't work for Buffalo but i do love there products enough to help people

atlupo

  • Calf
  • *
  • Posts: 12
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #2 on: October 28, 2014, 05:22:30 pm »
So i just confirming the new disk is in. If so, login to the UI of the unit and go to the settings where it shows the hdds/array, there will be a option in the array area to rebuild the unit.

Here is a guide on how to do it.
http://manual.buffalo.jp/buf-doc/35011719-02_EN.pdf

Thanks very much for the info. I was aware of the pdf you linked. However, I think the issue is now maybe "partially" resolved.

After my updated post I noticed that the drive tray for HD4 was slightly bent and that was preventing the new drive from seating properly in the SATA connector (I think). I un-bent the drive tray and the NAS now recognized the new drive and I got the message to rebuild the array. That's fine (working the way it should) but before I did that I just decided to try the old drive one last time and surprise, surprise, the NAS re-recognized it as part of the original RAID 5 array! So whatever caused the original failure somehow cleared itself!?

Problem now is I'm still getting "OperationModeI12 DEGRADE MODE" error message and I can't modify the RAID scanning settings (button greyed out). I ran a "check disk" on the array which showed no errors. I have the unit set to do a RAID scan once a month but I want to manually run one now; the GUI web access interface won't let me do that (button greyed out).

Do you know how to clear the error message and/or get back the ability to change the RAID scanning settings?

joma90

  • VIP
  • *
  • Posts: 565
  • Life would be easy if we only had the source code!
    • Let me show you how to google
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #3 on: October 28, 2014, 11:03:29 pm »
You can try rebooting the device. Usually they go away on there own but if it doesn't not you might have to either do a firmware update to clear the message. If none of that works then there might be a issue with the drive and the unit is not seeing it. Check disk is ok but it looks for smart failure. Sometimes a drive will pass smart check but have enough relocated sectors or repairable sectors to cause issues.
Units I own: TS-XL, TS-WXL, LS-WVL, TS-RXL,  TS5400, TS-RVHL,

I don't work for Buffalo but i do love there products enough to help people

atlupo

  • Calf
  • *
  • Posts: 12
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #4 on: October 29, 2014, 04:32:51 pm »
You can try rebooting the device. Usually they go away on there own but if it doesn't not you might have to either do a firmware update to clear the message. If none of that works then there might be a issue with the drive and the unit is not seeing it. Check disk is ok but it looks for smart failure. Sometimes a drive will pass smart check but have enough relocated sectors or repairable sectors to cause issues.

I have rebooted the device multiple times - no change.

I also "tricked" the device to run a RAID scan by manually setting the device time to just before the next scheduled scan.
RAID scan ran but finished much too quickly - usually takes ~ 4 hours - this was done in less than 1 minute; reported no problems but I still have error message ("I12 the RAID array is in degraded mode"). And I don't trust the RAID scan if it took such a short amount of time.

I am already at firmware 1.22. I attempted to re-install firmware but TSUpdater.exe reports no devices available for upgrade. So I modified TSUdater.ini to turn off version check flag ("VersionCheck = 0"). With this mod when I then run TSUpdater.exe it finds the device but when I try to upgrade firmware it doesn't seem to do anything other than re-displaying the dialog box. So, no luck with that approach either.

I'm still waiting on Buffalo tech support but they are rather slow in their responses - at least for this issue for me.

Anything else you can think of?

Thanks again for your feedback.

joma90

  • VIP
  • *
  • Posts: 565
  • Life would be easy if we only had the source code!
    • Let me show you how to google
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #5 on: October 30, 2014, 07:27:18 pm »
My bad here. I gave you the hard drive replacement pdf for a LS-QVL, This is the right pdf http://manual.buffalo.jp/buf-doc/35011966-1_EN.pdf, see if that works


Kind of forgot about the TS-QVHL unit. rarely see much about them. Also here is the main download page. http://d.buffalo.jp/ts-qvhl/
Units I own: TS-XL, TS-WXL, LS-WVL, TS-RXL,  TS5400, TS-RVHL,

I don't work for Buffalo but i do love there products enough to help people

atlupo

  • Calf
  • *
  • Posts: 12
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #6 on: October 31, 2014, 12:30:33 pm »
My bad here. I gave you the hard drive replacement pdf for a LS-QVL, This is the right pdf http://manual.buffalo.jp/buf-doc/35011966-1_EN.pdf, see if that works


Kind of forgot about the TS-QVHL unit. rarely see much about them. Also here is the main download page. http://d.buffalo.jp/ts-qvhl/

Thanks again. I had realized that your original link was for the different unit - and I had already seen the corresponding pdf for the TS-QVH12TL that you have included here.

In any case, I reminded myself that I had almost the identical problem 5 years ago with a TS-TGL (unit which I still have and is still functioning!) and posted here as well. As painful as that experience was I am surprised that I had initially forgotten about it. In any case, solution then was to dump all data off the unit, upgrade the firmware, reformat the drives, recreate brand new RAID array and load data back onto the unit. A real pain but that was the only thing that worked with the TS-TGL. And now, I guess, I will do the same with the TS-QVH12L.

Overall, I do like the units (have had several of the last 10 years or so) but now that I have been burned twice with a issue related to a simple hard drive failure (something which RAID arrays are supposed to deal with gracefully) I really have to think about whether I want to continue with these Buffalo NASs in the future. They really need to improve the software / firmware - having to start from scratch just to clear an error message is a bit ridiculous in my opinion.

joma90

  • VIP
  • *
  • Posts: 565
  • Life would be easy if we only had the source code!
    • Let me show you how to google
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #7 on: October 31, 2014, 02:09:39 pm »
Ah your raid table or raid array got corrupted. I thought of the idea and telling you to try what you said but i don't like saying it because alot of people get all butt hurt.. why did this happen. I won't do this... you don't know what your saying... etc.. etc.. lol not saying you would of just rather not find out.. last resorts is when ill mention stuff like this.. because in the end if people don't have backups of there backups then there going to be like all my data is gone and your responsible... ... even though ill tell them to backup there data first they seem to miss that part ... eh it has happened.
Units I own: TS-XL, TS-WXL, LS-WVL, TS-RXL,  TS5400, TS-RVHL,

I don't work for Buffalo but i do love there products enough to help people

atlupo

  • Calf
  • *
  • Posts: 12
Re: Problems replacing failed hard drive TS-QVH12TL/R6
« Reply #8 on: November 20, 2014, 10:23:02 am »
To all concerned - finally back to normal. Only solution was (as suggested in this forum and by Buffalo tech support) to start from scratch:
     remove all data to another server
     delete array, reset to factory default settings
     recreate RAID array
     copy all data back to new RAID array

Did find out in the process that there were, in fact, many bad sectors in the original HD4 - hence the original error messages. Disappointing since drive is less than 3 years old. Once HD4 replaced with brand new drive, RAID array was built and checked without issues. I am now currently copying all data back to this new RAID array.

Many thanks to all, and in particular joma90, for your help.

Some comments (mostly regarding how unhappy I am with the firmware / software of these Buffalo terastations):
     Be aware that the process of creating and checking a new RAID 5 array takes a long time! 12 TB configured as RAID 5 gives about 9 TB of available storage - this took more than 60 hours to create and resync the array! Don't really understand why since the disks were wiped clean - why does it take so long to resync a blank RAID 5 array!? And the Buffalo FW/SW simply does not give you any progress update - no way to tell how long the process will take or how far along you are. The only way I found was to use putty.exe to ssh into the root account and use the "cat /proc/mdstat" command. The output gives you a status update so that you can get a rough idea when the resync process will be complete. Surely the Buffalo software should be able to give you this info without you having to result to Linux commands!?
     There should be a way to check sectors on a single drive within the array. That would have saved me from having to do the above process twice. I did it once with the old HD4 and waited 3 days until the sector errors started showing up in the syslog. I shouldn't have to off load all data, and reset RAID array just to find out (3 days later!) that a hard drive might have developed bad sectors.
     And finally, there should be a way to manually clear error messages and/or re-run a RAID scan once they do appear. Once RAID errors appear, the button to modify RAID scanning settings became disabled. There is no way to get into that panel to re-run a scan on demand. I had to reset the date and time to fool the unit to doing a RAID scan at the next scheduled time. This would have also saved me some time. And even when I did that the scan finished so quickly I couldn't trust the result.

Sorry to say, I have used the Buffalo terastations for more than 10 years and when working fine they are very nice units, but when something does go wrong the you-know-what really hits the fan. And I find there is usually no graceful way of extracting yourself from disaster without offloading data, resetting everything, and reloading all data. When you are dealing with TBs of data this is simply not a desirable option. For my next NAS I will be looking at the competition - can't imagine myself going through this again in the future.