data storage and backup

I’ve been lucky…when my last hard drive started to go out, it made a slow whining sound. Luckily, I was able to get all the data off of it and onto another drive before it died. When I say all the data, I’m talking 80GB of digital pictures, movies, music, etc. in addition to all my school work from UC Davis and all my personal financial records and documents. After this brush with near-data-loss, I decided that I had better get smarter about my data management. I replaced the 200GB SATA drive with another 200GB SATA drive. This time, when installing windows, I created a 50GB partition on which to install Windows and all my applications and the remaining space I turned into a 150GB data drive. The idea was that if my windows install got too corrupted, I could leave all my data in place and reinstall windows on the 50GB partition.

This solution was fine. I was happy with it. One day, I was really thinking about how much it would suck if I permanently lost all my data. Think about it…I haven’t had any photos printed onto paper in almost 4 or 5 years. Everything is digital.

I began thinking about a backup solution. I had added a second 200GB SATA hard drive to my system. I decided to mirror my data on the second hard disk. I wrote a batch file to copy all the data from my music folder, my work directory (where all my programming and photoshop files are), and to backup my CVS repository, etc. I set up a scheduled task in the windows control panel to mirror the data every night at 4 AM.

Now, if one HD fails, at least there’s a backup copy of my data….but what about a fire? I would still lose it all in a fire. One cheap and relatively easy solution is to have an external usb HD that gets backed up each night that is kept near my bed. In the event of a fire, I could grab it and go. But in an emergency situation, that too could be too much. Faced with the decision of life or loss of data, I would much prefer life, but I would still hate to loose my data. I stumbled onto a page on the internet(thanks Al Gore) where a guy had an external network-attached hard drive inside a fire safe. The safe had electricity inside it. He used a couple devices that allow network traffic to be routed over power lines, so he was attached to his LAN inside the safe. The only problem with his setup was that the fire safe he used was not a Media Safe.

Fire safes only guarantee that the internal temperature of the safe will stay below the flash point of paper. A hard drive can be fried at temperatures of about 150C, much less than the flash point of paper. Media safes are specially designed and constructed to keep internal temperatures low enough to prevent media (floppy disks, cds, film negatives, etc.) from being destroyed.

It would seem that if I duplicated the guys effort, but used a media safe instead of a plain fire safe, I might be okay. The problem here is that a decent media safe is expensive, and I have yet to see one with an internal power supply.

This really got me thinking. What to do? I could use a commercial backup system, but that means that I’m paying out the nose for it, and then I’d eat up all my bandwidth on my cable internet connection during the initial backup…..I would have to upload about 60GB worth of data.

The solution: build a box who’s sole purpose is to keep a backup copy of my data. Put this box at a location different from my computers such that tragedy would have to strike both locations simultaneosly for it to become a true data loss.

One of Laura’s friends was getting rid of her old PC (1.3 GHz celeron, 256MB ram, 40GB HD). I offered to take it off her hands. Once I got the box, I put in an additional 80GB HD. Then, I downloaded Fedora Core 6 and burned the installation CD’s. I popped in the first one and away I went. I first tried the graphical installation….I got about 95% of the way through and it froze up. I did some reading online and found that this was a common issue with machines that were at the min. hardware specs. I restarted the installation, but this time in text mode. This time, I was successful. After installing FC6 and verifying I could log in remotely via ssh, I disconnected the monitor, keyboard, and mouse making it a headless box.

I then rewrote my backup scripts on my windows machine. I’ve been using a cygwin environment on my XP machine so that I could use *nix utilities like rsync. I set the scripts up such that they still mirror my important data from one 200GB drive to the other, but then added another script to move the data to my linux box. I generated some ssh keys so that I won’t have to be there to type in a password when the backup starts. I then did the initial backup (~60GB). All backups from here on out will be incremental backups, so only the files/directories that change will get downloaded/uploaded to the new box.

The final step in my backup script installation will be to setup a cygwin chron job to automatically run my backup scritps. I haven’t done this yet, but will soon.

Now, where to put the linux backup box? I haven’t decided yet. It will need to go to someone’s house who won’t mind it being there, turned on ²⁴⁄₇, and needs to be hardwired to their hi-speed internet connection. I’ll be polite and start my backups from my XP machine in the middle of the night so that I’m not robbing them of their bandwidth.

I still need to do a few things to complete the solution. I need to create a dyndns account for the linux box and install a dyndns client. I need to edit my XP backup script to point at this dyndns domain instead of the local IP from my network. I need to physically install the box at the chosen residence. I need forward the ports on their router to the box (22, maybe 80 if I decide to run some sort of a web interface off the box).

Anyways, in the event of a catastrophe at my apartment (fire, flood, earthquake, data loss, HD failure, EMP from a nuclear blast, etc.), my data will be safe at the other residence.