Guy Rutenberg

This tutorial will hopefully guide you in making automated encrypted backups to Amazon’s S3 using duplicity. It was written as a followup for Using Duplicity and Amazon S3 – Notes and Examples, in order to organize all the necessary information into a simple tutorial.

Will start by creating a simple wrapper to duplicity:

#! /usr/bin/python
import sys
import os
 
duplicity_bin = '/usr/bin/duplicity'
 
env = {
    'AWS_ACCESS_KEY_ID':     'PUT YOUR KEY ID HERE',
    'AWS_SECRET_ACCESS_KEY': 'PUT YOUR SECRET ACCESS KEY HERE',
    'PASSPHRASE':            'PUT ENCRYPTION PASSPHRASE',
}
env.update(os.environ)
 
os.execve(duplicity_bin, sys.argv, env)

save this under dupicity-wrapper.py and chmod 0500 it so only you will be able to read and execute it.

Note: You’ll want to write down the passphrase and store it in a safe location (preferably in two separate locations). So in case you’ll need to restore the backups, you won’t have useless encrypted files.

Now edit your crontab and add a line like the following:

10 1 * * 0 /path/to/duplicity-wrapper.py /path/to/folder/ s3+http://bucket-name/somefolder &>> ~/log/backups.log

This will create a weekly backup for /path/to/folder. The backup will be encrypted with what ever passphrase you’ve given in the duplicity-wrapper.py. The output of the backup process will be saved into ~/log/backups.log.

You should also run

/path/to/duplicity-wrapper.py full /path/to/folder/ s3+http://bucket-name/somefolder

in order to create full backups. You might want to periodically verify your backups:

/path/to/duplicity-wrapper.py collection-status s3+http://bucket-name/somefolder
/path/to/duplicity-wrapper.py verify s3+http://bucket-name/somefolder /path/to/folder/

To check the status of the backups and to verify them.

And last but not least, in case you ever need the backups, you can restore them using:

/path/to/duplicity-wrapper.py restore s3+http://bucket-name/somefolder /path/to/folder/

Security Considerations

As I know, some people will comment on the saving of the encryption passphrase plainly in a file, I will explain my reasoning. I use the above encryption in order to secure my files in case of data leakage from Amazon S3. In order to read my backups, or silently temper with them, some on will have to get the passphrase from my machine. While, this isn’t impossible, I will say it’s unlikely. Furthermore, if someone has access allowing him to read files from my computer, he doesn’t need the backups, he can access the files directly.

I’ve given some thought about making the backups more secure, but it seems you always have to compromise on either automation or incremental backups. But, as I wrote, the current solution seems to me strong enough given the circumstances. Nonetheless, if you’ve got a better solution it would be nice to hear.

Up until now I’ve been doing my backups to Amazon S3 using my s3backup script. While it’s simple and does what I needed at the time, I’ve decided to cut some of the costs by switching to incremental backups.
Read the rest of this entry »

This is an updated version of my previous backups script – Backup Directories to Amazon S3 Script. The new script works much better and safer. Unlike the old script, the new one creates the tarballs in a temporary file under /tmp, and allows more control over the backup process.

Read the rest of this entry »

This is an updated version of my WordPress Backup Script. The new version basically does the same thing: backup up a wordpress blog (actually any site that consists of files and a MySQL database). The new thing about the script is that instead of only saving the backup locally, it also uploads it to Amazon S3.

Read the rest of this entry »

This is a small script I wrote today, to automate my backups, which I do on Amazon S3. This is fairly short, yet useful bash script that utilize the s3cmd to do the actual sending of the files.
Read the rest of this entry »