So your desktop is backed up? Did you backup that server? Your backups are even on a different machine?

Not bad.

You even know to store a copy of your data in a geographically different location (“offsite backup”).

Excellent… Excellent start, but far from sufficient.

All the above are best practises – it’s a low bar, you need to do this, no questions asked. A failure to do so would be incredibly negligent and set yourself up for desaster. But it’s only half the story – if that: Lets not even talk about potential privacy issues or protection of your important data in case the unencrypted backup media are stolen, etc.

The real problem with backups appear when you have to restore them!

And that’s when you discover that:

  • The backed-up files were encrypted with a cryptographically-secure key, the only copy of which was on the machine that was lost
  • The Windows server had enormous amounts of configuration information stored in the IIS metabase which wasn’t backed up
  • Your accounting software wrote important data files in areas that weren’t coverred by the backup
  • Your Linux setup had vast amounts of organically grown SELinux configurations that were neither documented nor captured by your backup
  • The backup files were being copied to a FAT partition and were silently being truncated to 2GB
  • Your backups were on an LTO drive which was lost with the data center, and you can’t get another LTO drive for three days
  • And a million other things that can go wrong even when you “have backups.”

The minimum bar for a reliable service is not that you have done a backup, but that you have done a restore. On a new machine from ground up. If you’re running a web service, you need to be able to demonstrate that you can build a reasonably recent copy of the entire site, in a reasonable amount of time, on a new server or servers without ever accessing anything that was in the original data center. The bar is that you’ve done a restore.

We never ask our clients if they’re doing backups, we ask if they’re doing restores.