At home we have a 4-disk QNAP box as a file server which hosts photos dating back to the 1990s. Up to recently it was backed up over eSATA to external drives, but it was never a good solution. The QNAP box does offer cloud backup, but I don’t want to be dependent on somebody else’s proprietary way of copying data to the cloud. So, yesterday I finally got hacking on a Python script to back the whole thing up to a cloud provider.
After looking at Amazon S3, Google Cloud Platform and BLOB storage in Microsoft Azure (which I use frequently at work) I finally went with S3 as it has the option to automatically shift data to the ultra-low-cost Glacier service after a set time. There are good tutorials to get started for those who are interested here: http://boto.cloudhackers.com/en/latest/s3_tut.html
Amazon recommends splitting files larger than 100Mb prior to upload and Boto can be used with file splitting as well.
Prior to this it was necessary to encrypt all data to ensure it wasn’t easily accessible by any third party. Not that I expect anyone to have an interest in some family photos, but anyway. To make sure it would be possible even for my wife to decrypt the data I went with 7za since it simply creates zip files encrypted with AES. Encrypting is as easy as:
I may post the actual backup script here as well once it’s been through a few revisions, but it’s too rough for publication right now.