Trying to backup Terastation III's and Terastation Duo's...
The strategy we had in mind was that we could simply schedule Terastation - To - Terastation backups on a weekly basis and away we go.
Just learned from support that this option won't work @ speeds faster then 500 kb/sec even when connected to a gigE LAN (not even 1 megabyte per sec) yet writing to these boxes from computers in excess of 15 megabytes/sec...that's a bit of a limit isn't it?
Given our users won't have computers up and running, what is a practical way for us to accomplish these backups, as this is kind of killing the entire purpose for these systems...
Appreciate any help, thanks.
---
@Colin137
Yeah. We noted backup speeds from one system to another (through a cisco gigE switch w/ both devices attached) with backup jobs reporting the following:
"
sent 45457645798 bytes received 740048 bytes 406391.88 bytes/sec
total size is 60944558226 speedup is 1.34
"
Both systems were both TS Duo's in the recorded tests, but we've noted the same sluggish perf from TS III's.
406391.88 is where we get our real world 406Kb/sec speed, which is apparently expected due to a 500Kb/sec limitation so I guess 80% efficiency in general is good, but in this case, no go :(
I'll PM you my phone # if that helps look up the support inquiry (incase there's any reference there w/ regard to the last phone call).
Appreciate your assistance (incase there's another way we can accomplish the same thing, or if perhaps this is the limit).
Thanks!
Edit: PM sent, received, and 2nd PM send, thanks! :)
Sounds like someting that definitely needs to get worked out, whether is't a network issue at your end or a glich within the TS-III's.
But in the meantime, what about the USB backup option? I have my LinkStation Live backing itself up with a differential backup, to an attached USB drive every night at 3am. I get about 5mb/sec performance on backups to USB. This speed seems to have nothing to do with the performance of the USB drive itself; it doesn't matter whether i use my FreeAgent USB, my super-duper Hyperdrive Colorspace O in USB mode, or my generic USB enclosure with an old 160GB hard drive in it, cannabalized out of a bricked Linkstation 160.
I understand the appeal of a through-the-network backup; you can keep the devices separated by distances limited only by the physical layout of your network, to provide some measure of theft/fire protection, whereas a USB drive has to be connected directly to the TS-III. But for the short term until you get the network performance issue resolved, it may be a good fallback option. As a matter of fact, even if you get the network issue resolved, it never hurts to have a USB drive backup TOO.
Yeah, the only catch I have is if using usb-based backup, I can't set any access controls on the usb drive directly so i get to setup little locked cages, lock down the drives, etc. not the most fun, but if i gotta go that route temporarily until i get a different solution in-house, then it shall be.
If I can go a network device to another (filesystems w/ active directory controls in place) then i'm golden :)
@Colin137,
Much appreciated sir, thanks! As an fyi, here are the start + finish of our latest backup round from the backup log file:
-------- BEGIN BACKUP 2009/07/04 23:45:02 array1/shared -> TSJLF-DUO-BK@array1/shared-bk --------
sending incremental file list
created directory /200907042345
........(all files/directories listed here)
sent 585244712912 bytes received 2352365 bytes 881848.40 bytes/sec
total size is 598009426593 speedup is 1.02
-------- END BACKUP 2009/07/12 16:06:11 array1/shared -> TSJLF-DUO-BK@array1/shared-bk --------
Note that this time the backup job took 8 days w/ no other I/O going on.
Also this is with Compression enabled. So we're @ 881KB/sec. Still < 1 MBps though.
And yeah, LS Pro Duo's seem to behave faster based on what Ive seen as well. At least when 2 backup to eachother.
Again appreciate your help w/ this. Kudos!
---
I have a feeling the reason for this is the way that it calculates the speed for the log file... That's the average speed of the backup taken over the entire backup period. I really doubt that's the actual network throughput for the file transfer.
How many files are being transferred?
~100000 files.
What I can say, the same dataset on an LS Pro Duo going to another LS Pro Duo takes ~4 days to complete, whereas it's significantly longer on the Terastations. (~ twice as long).
Also moving off of TS through a CIFS (samba)/windows share, takes < 1 day to copy it to local drives, and it takes < 1 day to move it to an alternate Terastation (again assuming i move the files through a Linux or Windows system).
Appreciate your help sir, thanks!
There's something very odd going on here...
Just completed an append backup from TS III to TS Duo, total size was 227 GB, data transferred was about 180 GB.
-------- BEGIN BACKUP 2009/07/15 14:42:01 array1/backuptest -> TS-WXL4B5@array1/backuptest --------
sending incremental file list
<SNIP />
sent 184057238277 bytes received 791336 bytes 15136973.53 bytes/sec
total size is 227486776007 speedup is 1.24
-------- END BACKUP 2009/07/15 18:04:51 array1/backuptest -> TS-WXL4B5@array1/backuptest --------
As you can see, I'm getting a blazing 14-15 MB/s... much better than even the Pro Duo. This is what I was expecting, what with the faster processor and all. No transfer cap in sight.
Had my switch not decided to mess up, the first time i tried would have been a full backup. I'm deleting stuff off the source and starting it again. This brings me to a question I should have asked much earlier...
Can you post the settings that you have configured for the backup?
Sure, here are the settings. My apologies for having not posted these sooner, my bad sir.
--
Backup Operation Mode: Normal Backup
Create Backup Log File
Use Compressed Transfer Method
Ignore Errors and Proceed with Backup
--
Some other things that might be worth noting:
Array 1 on both source and destination are Encrypted (via the encrypted setting on the Terastation). The backup encryption option is -NOT- selected though, only the arrays themselves.
We're goign from a R1-mirror to a R1-mirror...
We have APC UPS's connected to the devices with Enabled Status and Normal status.
HDD Spindown is enabled on both devices. Both are running 1.04 firmware.
Looking at one of the systems, the 2 drives making up the mirror are WD10EADS (came w/ the systems, have not replaced or added anything).
The NIC frame size is: 1,518 bytes(Default) for both devices.
We've tried connected to the same Cisco 4948 (GigE) switch, and also to the same WRT600N (Linksys wifi + wired GigE, both on the GigE) with almost identical results.
Again, appreciate your help w/ this sir! :)
@Colin137,
Gotcha. We estimated we'd get a worst-case of 80% reduced performance w/ the encryption overhead, but ~500KB/sec seems a bit excessive...at least to me it does.. :)
Also thanks for invaliding the "500KB" info we received. One bit of feedback would be to make sure the education of this gets out there, as hitting a theoretical 500KB limit for backups is a huge deterrent, and I'm glad to see it's not the true case! :)
Thanks again for your help, and look forward to your findings.
@Colin137,
Thanks sir. Yeah, even a 50% reduction in total performance would be acceptable + a huge step in the right direction from where we're at, but we definitely don't have small datasets.
Appreciate the update and look forward to your findings. Thanks again sir.
---
@Colin137,
Sir, just checking to see if you had any update relative to performance for large datasets w/ encryption enabled?
Thanks sir,
---
@Colin137,
No worries. Will stay tuned. Thanks sir.
---