Thursday, November 22, 2007

Half a Terabyte, Hold the Gravy

What do you do when you've got a device that creates very valuable 2MB files with the press of a button, an action that you may repeat up to 600 times an hour (maybe 20GB a week)? You end up buying another device, such as this puppy: the 500GB Lacie D2 HD Quadra 7200RPM 16MB external drive. I had never before heard of LaCIE, a French concern, but this drive seemed to fit the bill as a backup device. It has a reasonable dollars per gigabyte number, the reviews seemed encouraging, and I went ahead and paid the extra $60 to get the Firewire-capable model as this seems to be recommended over USB for sustained reads and writes typical of an external drive.

PriceGrabber got me thinking about the whole backup plan from the dollars per gigabyte angle for different types of storage devices, summarized here:

Tech Sample Product Price w/ S&H Capacity $/GB
DVD-R Verbatim DVD-R 16x 100-pack $42.27 470GB $0.09/GB
external HD 500GB Lacie D2 HD Quadra 7200RPM 16MB $182.60 500 GB $0.36/GB
flash memory PNY 4GB USB flash drive $31.89 4 GB $7.97/GB
DVD's are an economical choice as a backup medium, especially in my case where making modifications is not important. The 100 disk spindle is almost a direct comparison to the LaCIE drive in terms of capacity, since the LaCIE's actual capacity is more like 460GB. The external drive has the advantage that you don't need find the right disk when you need to retrieve your backup, so it is a good "live" backup solution. The external drive also gives you a little bit of portability, and the model I bought has four connection technologies (hence the name "Quadra") giving some flexibility in working with other devices.

The USB flash drive, on the other hand, is a terrible choice for long term storage, as far as I can tell. That salesman at the electronics store certainly had a lot of nerve trying to talk my Dad out of buying blank CDs in favor of buying a flash drive. The advantage of flash drives is portability, not economics. They weren't looking much cheaper than $8 a gigabyte on PriceGrabber for 1GB and up.

I've left out some options here which I didn't research as much. I've left out the venerable tape backup as well as internal hard drives and NAS. Internal hard drives in a RAID might be a good alternative to the single external drive because you get some automatic failure detection. And the NAS, or Network Attached Storage, goes one better in simplicity.

For this exercise, I've included in my personal backup plan ditching the idea of an "archive quality" medium such as tape in favor of any storage medium which has a long enough life to copy the data onto something newer.

Another idea is to build layers of redundance; one backup is not good enough, and maybe not even two. Somewhere in the back of my head in all this is my dear Grandma's judgment of disgust at the idea of computers, where you can erase everything at the press of a button. This information vulnerability is certainly an Achille's Heel of computing, and it's been an interesting exercise to price out and plan a moderately serious backup system. I'm certainly a novice to this, so if you have some experience to share, I'm all ears.


Susan's Husband said...

Heh, "archive quality". In real life, the hard disk is probably much closer to archive quality than tapes. Tapes are in fact very flaky with serious reliability problems. Trusting any data to a single tape is asking for failure. A hard disk, on the other hand, is very stable if it's turned off. Probably much more stable than a DVD or CD-ROM as well.

I know of people who've migrated form tape to extern hard disks because they view it as a clearly superior choice, not only more convenient and more reliable, but quite possibly cheaper as well. With something like Ghost, which copies and compresses the entire live drive contents you've got quite a nice backup system.

If I were you and those files were in fact valuable, I would get a desktop RAID for main storage and use external HDs for long term backup.

Matt Olson said...

If you're not gonna keep these files forever, but they're truly valuable, and if you have an internet connection, you could look at Amazon's S3 service. Their servers let you store data for $0.15/GB/month, and it's encrypted and redundantly stored in datacenters all around the country. Pay $20 upfront for JungleDisk (which I'm looking into trying right now) and you can map your S3 account as a drive in Windows, Mac, or Linux systems. Of course, the longer you want to keep stuff, the less cost-effective this becomes...

JFKBits said...

Susan's huband: Tape backup is something I didn't have as much hard data to share, so I didn't elaborate in the post, but I'd rejected the idea based purely on the pitiful capacities of tapes I'd seen in a quick search. The "archive quality" tape backup I was thinking of was described to me by a system administrator from one of the big US government labs where he oversees a huge, gigantic scientific data collection. His vendors were telling him numbers like >100 years. Of course I don't believe those numbers, but even if it's half that long, I also don't imagine it's in my price range.

JFKBits said...

Matt olson: yes, Amazon's S3 was something I looked into closely (thanks for the mention). It didn't work for my needs because I wanted storage for an indefinite length of time. The big win with S3 is having someone else administrate the hardware, not to mention the space savings.