A Pox on all NAS devices!
Monday, November 9, 2009 at 8:11PM Moores Law states that CPU power will double every 18 months and half in price. Hard drives havent really kept up with this rate of performance increase (and I'm not rich enough to buy SSD yet), but have came down in price. Our local Tesco's for instance, sells 1tb USB hard drives for £65. Very cheap. And now you have to consider that just about every household out there have these drives kicking around. Some households have multiple computers and currently use wifi for networking between them.
So how do you back up your 300gb Music collection, your 100gb photo collection, and the two machines on your network ?
Slowly.
Even the 'plug-in-the-USB' port devices take over 15 hours to copy 1tb of drive. Currently, I'm moving stuff from older USB drive storage in the back of machines into NAS devices:
- A Netgear ReadyNas Duo Device (running the latest firmware) with two fast 1tb drives in it
- FreeNas - a free NAS set of utilities, hosted on one of my VMWare ESX 4i servers.
I'm looking at 4 days to copy 1.5tb of stuff around. Honestly!
Surely there's faster solutions around ? Or do I just rip the drives out of the NAS devices and put em back into the Windows 2003 servers I have ?
Extra points for a Mac AND PC solution (three Macs in this house, one Vista PC, one Netware 7, one XP, and four Windows 2003/8 servers)
Reader Comments (15)
I have been very happy with my Windows Home Server. The HP ones work with Apple and Windows PCs.
Trying SATA drives out for that here shortly. New and old laptop have SATA plugs so it will be hopefully fast and easy to copy all stuff on the 1 TB external drive. That one will eventually be added to the linux server (if Linux takes the SATA card....didn't really check...lol...)
When I bought that SATA drive (too long ago to admit) it seemed so much faster than USB... I forget...
Meant to add, the WHS is also Gb network. All my machines are getting incrememental backups each night.
And kind of like DAOS, files that are common to each machine, only have a single copy stored.
Have you tried an e-sata enclosure?
I bought a NAS a year or so back that claimed to have a Gbit network port. Someone on the vendor's official forum complained that this product couldn't even manage 100 Mbit, and the response was that it *does* have a Gbit network port, but the device's CPU isn't capable of that network speed.
Is this sending all the bits over the network? Can you plug the external hdd into the nas? Does copying then still send everything over the network? Can you ssh into the nas and copy from there?
Somethings not right with 4 days to copy 1.5TB.
One thing to remember is that USB uses CPU to run and therefore can't run at top whack if the CPU can't keep up. Firewire doesn't have that problem so can run at full speed all the time. I'm very happy with my FW800 drives. Just a shame they cost so much more.
My plan is to get a FW800 hdd cradle for backup to cheap internal drives that don't need to be online and can sit on a shelf.
Bill
The NTFS driver on the ReadyNAS sucks, that may be causing unnecessary slowdown.
{ http://www.readynas.com/forum/search.php?keywords=NTFS+slow&terms=all&author=&sc=1&sf=all&sk=t&sd=d&sr=posts&st=0&ch=300&t=0&submit=Search" rel="nofollow" target ="blank">Link }
There are many other reasons why performance could be sucking in this case, but that one seems to be the most likely.
There is a whole section devoted to it if you have the time and patience...
{ http://www.readynas.com/forum/viewforum.php?f=21&sid=23eec6d6068892466dc8f83c13915b79" rel="nofollow" target ="blank">Link }
Good luck!
thanks y'all. I'm running gigabit - but yeah - its all going to very slow atom-based CPU's in the dedicated devices.
I might just bite the bullet and buy a new home server sometime.
---* Bill
Best solution is to not have to copy everything all at once.
Set up a good NAS with some kind of redundant RAID system, perhaps RAID-5. Once you get your data there, just access it from there. If you add more data, put it on the NAS. If a drive in the NAS fails, replace and let it rebuild.
In Wikipedia
{ http://en.wikipedia.org/wiki/RAID" rel="nofollow" target ="blank">Link }
"BeyondRAID, created by Data Robotics and used in the Drobo series of products, implements both mirroring and striping simultaneously or individually dependent on disk and data context. It offers expandability without reconfiguration, the ability to mix and match drive sizes and the ability to reorder disks. It supports NTFS, HFS+, FAT32, and EXT3 file systems[6]. It also uses thin provisioning to allow for single volumes up to 16 TB depending on the host operating system support."
Okay, maybe overkill, but I do a lot of video editing and have about 6TB of data. I've been looking at linear tape-open (LTO) as a possible way to archive this quantity of data. But it isn't cheap.
{ http://en.wikipedia.org/wiki/Linear_Tape-Open" rel="nofollow" target ="blank">Link }
eSATA is without question the way to go. get an enclosure for around €25 and it'll have usb2 as well as eSATA. The hot pluggable bit is not something I'd use but the speed is the same as internal SATA.
@9 the biggest risk in a home situation is theft of the device and the backup device - it is a bigger risk than raid failure. Hence why it is good to use some cheap usb drives or similar as well as the NAS
@11 indeed. I use a ReadyNAS in the home office, but I also do off-site back-ups with Jungledisk, which leverages the Amazon S3 infrastructure. Nifty stuff, reliable and cheap!
Bill, I was struggling with this same problem and come up with a different solution.
First, I copied all my key from each workstation to my (Windows) server. So at that point my workstations and server were in sync.
Then I signed up with SpiderOak for cloud backup. It's Mac/Windows/Linux, which is key for me. It also allows me to sync directories via the cloud, and access my data anywhere via a browser.
I installed the client on all my machines and told the server and clients to keep the key directories in sync. Now SpiderOak runs in the background and synchronizes changes between workstations and server, in the background.
Last step is to use a batch file on the Windows server to nightly copy the data to a USB drive. So I have both on-site and off-site bnackups.
Drawbacks? Yeah, some.
1) The initial upload and sync can take a LOT of time.
2) Gotta pay for the storage you need. So you have to be selective about what you back up to the cloud. But $100 per year for 100Gb isn't bad.
3) SpiderOak is still a maturing product, so the web site is not as full-featured as you might like.
But for me, the advatages - cross-platform sync, background operation, and off-site backup - are worth it. I no longer have to fear the wrath of SWMBO should her Mac with all of our family photos disappear, even if the house burns down.
eSATA support on macs is a little on the ropey side. I wouldn't trust it until things improve.
Bill,
I've had this issue at home for a while now. I looked in depth at all the reviews of the "budget" NAS systems including the Buffalo systems. The result was not good. Best I could see was 12mbytes/s read or write sustained over a gig network.
Not quite USB speeds was it???
The main comment was that the very slow CPU's used in the NAS boxes simply couldn't deal with the Gigabit network card and couldn't feed it fast enough.
I did some experimentation with a spare machine at home (xp2600 1 gig ram). I tried using Freenas on it but it simply didn't want to play with the drives. Way too many issues.
I have it on XP at the moment and it's performing OK, but not too well. My main issue is that the gigabit network cards in both my main workstation and the xp2600 machine (add in card), won't work with my netgear gigabit switch. So my best experience was with a billion router acting as a hub (not optimal). I managed 55mbytes/s burst and 35mbytes/s sustained in that setup.
However my all time favourite method of doing NAS, and given that you have an ESX3.5i server this should work for you, is to create a freenas VM on ESX. Add the freenas CD to the setup and Add a blank floppy image to it on the ESX server.
It worked flawlessly for me for 6 months and was every bit as fast as the Windows servers I was copying to/from (if not a bit faster).
Of course it's not portable but we can't have everything can we........