Disaster Recovery

Testing Methodology

While a NAS appliance should be easy to use and have decent performance there is one area where it absolutely has to be 100% reliable and that is with its ability to recover from a dead hard drive (aka Disaster Recovery). Unfortunately, in the real world things are rarely as black and white as this and nothing should be taken for granted; especially when precious records are on the line. Even more basic is that while yes a NAS should always recover from a bad drive, but what about its rebuild times? The amount of time it takes for a NAS to rebuild its array is very critical, as this is a time where it is most vulnerable to catastrophic loss of data. This test can therefore be considered a three part test. The first part is PASS/FAIL and deals with did the RAID array work, is the array’s data still available and not corrupt when a hard drive “dies”. The second part deal with how the array deals with replacement of the missing hard drive and is also a PASS/FAIL in that does the array start rebuilding or will it crash / hang? Does it require you to rebuild the array manually by first destroying all data on the array and starting from scratch (which is in our opinion unacceptable performance). The third part (should it pass the first two parts)deals with how long it took for the array to get out of degraded mode and is also a PASS/FAIL. Regardless of performance we will inform you how long it takes but if its rebuild performance is less than 50 gigabytes an hour it is not fast enough to be considered effective.

To this end for each mode that the NAS offers that has disaster recovery, we start by first formatting and then filling them to the maximum capacity available in a given mode. After this was completed and we then verified that the data was indeed stored on the NAS before simulating a catastrophic failure. This failure was accomplished by removing one of the hard drives, while the unit is running (if it supports hot swapping, if it does not we power off the unit and remove the hard drive and then turn the unit back on). We then checked to see if the data was still available. If it is we then replace the “dead” hard drive and observe what the unit does with it. As soon as the hard drive is reinserted we start the timer and keep it running until the NAS informs us that it has finished its rebuild. All tests were run twice. All tests were not performed “one after the other”; rather they took place over a period of a week and were widely spaced out so as to help reduce any possible chances of overloading the array and introducing any non repeatable errors. If for any reason, the array fails a section of the test. The test will be reran, if it fails a second time it will be considered a true FAIL, we will however inform you of any errors we get during testing even if the error did not reoccur.

Test Results

RAID 1:

In RAID 1 the enclosure's hard drives showed as one single drive. When one of the hard drives was removed all data was still secure. The only noticeable difference was that the HDD1 Failure light went on. By reinserting the now reformatted second drive the enclosure immediately started to rebuild its array. Rebuild time took a few minutes over 5 hours to complete. Overall, this is acceptable performance (if just barely) for a NAS in this price range and can be considered a PASS on all 3 sections.