Author Topic: SLOW replication with MULTIPLE temp files for each 10 GB backup file  (Read 3121 times)

NWTech

  • Calf
  • *
  • Posts: 8

 

As the title says we have two terastions setup to replicate.

 

It does work (kind of) but is very slow, with multiple "temp files" showing up for each file duplicated

 

Files are 10GB backupexec bkf files, and it looks like it replicates each one at least three times before moving on to next one. At any time it has 6 of these temp files going at same time, which makes it very slow, and never seems to finish

 

With Tech Support help, we've upgraded to latest software and factory reset both terastaions to no avail

 

Any advice?

 

 


Chadster

  • Calf
  • *
  • Posts: 1
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #1 on: October 07, 2010, 04:09:28 PM »

Well, I had the same problem and decided to give up on the replication.  We were trying to do it between locations and it was too painfully slow.  I was told by tech support tha the larger the files are that I'm trying to replicate, the less effective the replication is.  I'm backing up SQL Server databases, so my files are pretty large.

 


davo

  • Really Big Bull
  • VIP
  • *
  • Posts: 6149
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #2 on: October 07, 2010, 05:32:28 PM »

How slow is slow exactly? Are the two TS's on the same LAN? Are they at different locations?

PM me for TFTP / Boot Images / Recovery files  LSRecovery.exe file.
Having network issues? Drop me an email: info@interwebnetworks.com and we will get it fixed!

NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #3 on: October 08, 2010, 07:07:42 AM »

If I "drag and drop" a 10GB file, it takes only a few minutes to copy fron one Terastation to another (we have a 10GB fiber backbone so it's pretty quick).  The source Terastation has a little over 3TB on it, and a fresh replication attempt (erasing all old files on target and starting over) has been running for nearly a week and isn't quite to 2TB

 

Yes, same flat lan architecture (no vlans, subnets, or other differences)

 

Different buildings (got flagged by an auditor for having backup only at one location) but all behind same firewall (no port filtering etc)


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #4 on: October 12, 2010, 07:04:56 AM »

"bump"

 

Sorry to be a bother, but I really need some help here and Tech Support has tried everything they can think of


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #5 on: October 12, 2010, 11:50:14 AM »

Rebooted both stations.

 

On TS target, it now shows the previously replicated files, with their original sizes dates etc as expected.

 

I restarted the replication (E26 when rebooted) and it shows only one copy/temp file in progress. Seems better so far.

 

But it is replicating a file that is already there from replication.

 

So it's time and bandwidth are not being used effectively, its being wasted overwriting existing replicated files

 

Anything to check??


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #6 on: October 14, 2010, 07:12:56 AM »

Now it's back to 5 temp files copying at once (4 duplicates of an already transferred file(!) and one other duplicate of an existing file) so it's not making any progress on catching up with replication from the source TS

 

I think this thing is posessed!!

 

Any advice?


Dustrega

  • Big Bull
  • *****
  • Posts: 1394
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #7 on: October 14, 2010, 03:10:34 PM »

Well, the transfer is probably a little slower than drag and drop because of the fact there's file checking.  Also, since the timestamps might be an issue I would check the time settings (including the GMT) and make sure those are sychronized.  Otherwise it's possible the TSs are fighting over which file (if one already exists) is newer.  Just a suggestion, could you post the firmware versions of both units per our forum guidelines?


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #8 on: October 15, 2010, 07:31:16 AM »

Firmware on both TS's is 1.41 (Tech Support had us upgrade both to newest version and do a factory defaults reset)

 

Both of the TS's had the same (but not correct) default timezone (must have changed during the factory reset). Set them both to EST and restarted the TS's

 

I completely understand the replication taking longer than drag and drop, but 2 weeks and still not to 2TB...

 

It seems like the duplicated effort is what is killing the replication progress

 

Check out the screenshot  , it shows the same file being copied several times at once, along with another file being copied, both of which are already there

 

Thanks for the reply, any more advice?


Dustrega

  • Big Bull
  • *****
  • Posts: 1394
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #9 on: October 15, 2010, 02:05:17 PM »

All of these backup files are not being updated by their respective workstations are they?  Also, if you tried turning on Jumbo Frames in the Network settings I wonder if this would improve the replication results.


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #10 on: October 15, 2010, 02:21:36 PM »

These are Backup Exec server backups that occur late at night, the jobs run Weekend - Full, and Weeknight - Append.  They are over well before start of business day. Once the 10gb files are created, they seem to remain the same until Backup Exec cleans off the oldest "media" (the files on the TS) to make room for newer backups

 

Happy to try "Jumbo Frames" settings if you think it would help


NWTech

  • Calf
  • *
  • Posts: 8
Re: SLOW replication with MULTIPLE temp files for each 10 GB backup file
« Reply #11 on: October 29, 2010, 08:53:24 AM »

With Jumbo frame sizes (tried all different options) we get an Replication Error 26. Only seems to work at default size

 

However, I believe I did find source of the problem.

 

Backup Exec runs a nightly append (and weekly full backup) to our source TS.

It reclaims disk space by overwriting the oldest "media" (the backup files).

It seems that the replication sees the file changing when these files are being over written, but because it so large (10GB) and takes a while to backup, it sees the file changing more than once so it starts copying the file repetitively.

If I reboot both stations in the morning, and restart replication, it works properly until the backup begins again that evening and the files get over written, which triggers the duplicated replication problem.

 

Quite simply, the replication process is not smart enough to stop copying the file when it sees it an updated version of the file of the same name (my humble opinion it should delete the temp file and start replicating the file over again, not waste time and bandwidth making duplicate copies)

 

I called tech support with this bit of deduction, looking for a glimmer of hope, and for the very first time I was very disappointed with the level of support I received. The woman I spoke to was not helpful at all, and basically said it was our problem that our files were being over written during replication. Is this not what happens to all files on any replication source TS??

 

Anything to try before I scrap replication?

 

Very frustrating for us, as we laid this scenario out in detail to pre-sales tech support who told us this would be "No problem at all"