Wednesday, December 7, 2011

BackupPC, the best backup I've worked with.

I am fortunate to work in a large company that uses macs for their workstations. All the shiny gui simplistic user facing stuff aside, under the hood it's BSD Unix, mostly. Sure, Apple has done their best to break Unix, and sometimes they succeed in even bringing down the kernel, but otherwise, it's good enough.

Being Unix, you have wonderful tools like rsync, ssh, and Gzip for easily moving data between machines with compression. You can build your own cron driven backup scheme if you really want to bang out the scripts, or you can run BackupPc. (link at end of post)

In my company, we need to maintain a backup of executive machines and some server data.

This is a task that could be quite expensive in most enterprise. A powerful server, running an expensive OS from M$. An equally expensive backup program, more money for licenses for the number of clients you'll be backing up.. An expensive DLT tape system or Optical based system. Maintenance heavy with downtime for M$ updates or (this always makes me chuckle) defragging the servers drive for performance..

In my case, a bit of ebay browsing netted a Dell PowerEdge 2950 server with 4gig of Ram, dual 2.3Ghz dual-core Xeons, and six SAS drive caddies, for only $400.00 Add six 1TB Western Digital green drives for not quite $800 and you're done with expense.

Now we build a RAID5 array with the drives and we're sitting on 4.5 TB of redundant storage. With compression, that's enough for over 100 client machines easy. (more on this in a moment)

Hardware acquired, lets look at software. I like Debian as an all around swiss army knife server OS. Everyone has their favorite, we're not going to debate who's OS can beat up who's here. Now, lets think about the discs. We want at least three partitions on our big RAID, one for the system, swap space, and our backup storage.

I set up about 40Gig for the root system, that's probably overkill by a long ways, but who knows about what might come down the road over the life of the machine? Swap space, I like to match my RAM size, so 4Gig there, and the rest as an ext3 partition mounted where BackupPC stores its backup data:

After formatting the partitions, Debian took only two minutes to install. Two minutes? Yes, that's all. I used the textual installer, and only checked the components I needed, apache, ssh server, base system.. No Gui needed on a real server! That would just waste resources.

Configuring BackupPC takes editing a few perl scripts. That might sound daunting, but they are HEAVILY commented, and it's not very hard to understand them. The tricky part is the clients.

For BackupPC to work best, you want to let it use rsync as it's method to move data from the clients. For rsync to work best, you want to exchange RSA public keys between the server and the clients. Since my company uses Unix based Macs as workstations, this is simple to set up. Now BackupPC has root access to the clients and can use rsync to pull data efficiently.

Internally BackupPC does some nice tricks to save disk space. For each client it backs up, it will look at the files in the backup pool, and remove any duplicates, replacing them with hard links. This is a HUGE space saver in an enterprise where many machines have the same applications installed. Additionally, backup data can be compressed on the fly with gzip, further saving space.

In my example, I have 30+ full workstations backed up in less than 1TB of space. More impressive when you count that one of the machines in particular has an insane 780Gig of data on it. (I won't comment on the why, it's not under my control.) most of the machines count toward 20-40Gig of data being backed up. Keep a week of incremental backups and a couple of full backups and I get up to 1.4TB. Still not bad at all.

On this older Dell, the performance under linux is so good that the 30 clients get through an incremental backup in just a few hours at night.

BackupPC is driven by a web interface that makes it very easy to monitor it and so so very easy to restore from the backup.

If a user needs a file, folder, or even their entire home directory restored during disaster recovery, it's quick and easy. In the web interface, you browse through their backups to find the file, folder, etc., then simply check a box next to it and click a button to restore the files. BackupPC then uses rsync again and puts the file right back directly to the clients machine. In seconds.

It's a beautiful thing.