Backups For the Technically Inclined

There are two kinds of people, those that have already had a hard drive failure and those that will have a hard drive failure.

I lost hard drives and data back in 2006 and I’ve been properly paranoid about backups ever since. I care about my data. I care about my photos, about my code, about the various notes/tasks/etc. that comprise my “digital brain”. I’m co-founded BackupMoxie because there wasn’t a backup service that met our needs.

A few important rules to make sure you can trust your backups:

  1. Take responsibility for your backups (don’t blindly trust a 3rd party).
  2. Test your backups to make sure they have the data you expect.

I use a single laptop computer which I connect to an external display/keyboard/mouse at home and work. At home I have a thunderbolt-connected drive (with 6 physical disks in a RAID 5 configuration) for media and other large files that don’t fit on my laptop. I have a ReadyNAS device (with 4 physical disks in a RAID 5 configuration) on our home network which acts as a media server and backup device.

I maintain two types of backups. The first is a complete copy of my laptop so that I can quickly replace it and get back to work if I need to. The second is for making sure I don’t lose data that is important to me (email, code, documents, photos, videos, etc.).

A good backup system does two things:

  1. makes sure you don’t lose data
  2. reduces “downtime” when you have a hardware failure

and has no single point of failure. Hard drives will fail, all of the devices in a single physical location could be wiped out or stolen… you want to be able to weather these things.

Backup Process Diagram

I use SuperDuper to create a full, bootable copy of my laptop on a sparse disk image file on my thunderbolt drive. I actually have two of these, and I alternate which one I update on a nightly schedule (using the scheduling feature of SuperDuper). The idea is that one of these may get corrupted, or that I might back up bad data to one of the backups, but hopefully that wouldn’t happen with both of them. In order for me to lose my laptop configuration, files, etc. I would need to physically lose the laptop and thunderbolt drive, or have the SSD in the laptop and multiple physical drives in the thunderbolt array fail simultaneously.

As for my files, I have most stuff1 on my laptop (some in Dropbox, some not) and I maintain a full copy of my documents, digital photos and videos on the thunderbolt drive. These are then copied via rsync to the ReadyNAS device on the local network. I also store a copy of these in Amazon Glacier, satisfying the need for off-site backups for these files. I use Arq to send the files to Glacier. My email and code live in various hosted services (FastMail, Gmail, GitHub, SVN, etc.) which act as an additional off-site backup for those items.2

With this configuration I have an easy recovery path should I need to replace my laptop, and have redundant protection for my data. And I sleep better at night.

I put together this post because a friend recently told me about how she lost 3 months of photos when they didn’t come back from iCloud and asked me how I do backups. I have heard countless similar stories of iCloud failures, Time Machine failures, Crashplan/Carbonite/Backblaze/etc. failures.3

I’m working on a recommended backup system for the less-technically inclined. It’s a challenge to walk the line between convenience and diligence. Once I have something I feel good about, I’ll post it here.

If you’re a photographer, musician, etc. and deal with lots and lots of data, be sure to check our how Eric Cheng has simplified his backup strategy.

  1. I don’t keep full-size/RAW photos from 1999-2010 on my laptop, just the current and previous year’s photos. I do have a full set of lower-resolution photos on my laptop, and also sync those to my mobile devices. 
  2. I pass my email through a Gmail account to FastMail account for additional redundancy. 
  3. Some are probably due to user error, others to technical failures – the outcome sucks in both situations.