Tag: duplicity

Duply credential error when using Amazon S3

Duply is a convenient wrapper around duplicity, a tool for encrypted incremental backups that I’ve used for the last couple of years. Recently, after an upgrade, my Amazon S3 backups failed, reporting the following error:

    'Check your credentials' % (len(names), str(names)))
NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. ['HmacAuthV1Handler'] Check your credentials

Boto, the backend duplicity relies on for the Amazon S3 backend, requires authentication parameters to be passed through the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. Because different backends require different variables, duply used to make that transparent: you would just set TARGET_USER and TARGET_PASS, and duply would take care of the rest. However, duply 1.10 broke compatibility and requires you to set the variables yourself. Hence, the fix is to replace the TARGET_* variables with exported AWS_* variables:

# TARGET_USER='XXXXXXXXXXXX'
# TARGET_PASS='XXXXXXXXXXXX'
export AWS_ACCESS_KEY_ID='XXXXXXXXXXXX'
export AWS_SECRET_ACCESS_KEY='XXXXXXXXXXXX'

Incremental WordPress Backups using Duply (Duplicity)

This post outlines how to create encrypted incremental backups for WordPress using duplicity and duply. The general method, as you will see, is pretty generic, and I’ve been using it successfully to back up Django sites and MediaWiki installations as well. You can use this method to make secure backups to almost any kind of service imaginable: ftp, sftp, Amazon S3, rsync, Rackspace Open Cloud, Ubuntu One, Google Drive, and whatever else you can think of (as long as the duplicity folks implemented it :-)). If you prefer a simpler solution, and don’t care about incremental or encrypted backups, see my Improved FTP Backup for WordPress or my WordPress Backup to Amazon S3 Script.
Continue reading Incremental WordPress Backups using Duply (Duplicity)

Gmail backup: getmail vs. OfflineIMAP

I’m currently reviewing my backup plans and decided it’s a good occasion to finally start backing up my Gmail account. Firstly, I didn’t seriously consider desktop clients as the main backup tool, as they are hard to automate. The two main options are OfflineIMAP and getmail. Both are available from Ubuntu’s repositories, so installation is easy with both, and both have good tutorials: Matt Cutts’ getmail and EnigmaCurry’s OfflineIMAP.

OfflineIMAP claims to be faster, but I haven’t really checked it (and I’m not sure how important that is, given that it runs in the background). From what I saw, configuring them is mainly a task of cut-and-paste, but getmail requires you to list every label you want to back up, which I consider a major downside. As both are able to save the mail in maildir format, it should be easy to back it up using duplicity.

Conclusion: This was a short comparison, mainly to guide me in choosing the right backup for me. You may have different opinions (which, of course, I would gladly hear). I finally chose OfflineIMAP, mainly due to the labels issue.

Note on desktop clients: It seems that every decent one can be configured to work with a local maildir, so you can use them to read the backups. As I prefer Gmail’s interface, I will only use desktop clients in case I’m offline, so read-only access from a desktop client seems good enough for me.

Automated Encrypted Backups to S3 Using Duplicity

This tutorial will hopefully guide you through making automated encrypted backups to Amazon’s S3 using duplicity. It was written as a follow-up to Using Duplicity and Amazon S3 – Notes and Examples, in order to organize all the necessary information into a simple tutorial.

We’ll start by creating a simple wrapper for duplicity:

#! /usr/bin/python
import sys
import os

duplicity_bin = '/usr/bin/duplicity'

env = {
    'AWS_ACCESS_KEY_ID':     'PUT YOUR KEY ID HERE',
    'AWS_SECRET_ACCESS_KEY': 'PUT YOUR SECRET ACCESS KEY HERE',
    'PASSPHRASE':            'PUT ENCRYPTION PASSPHRASE',
}
env.update(os.environ)

os.execve(duplicity_bin, sys.argv, env)

Save this under duplicity-wrapper.py and chmod 0500 it so only you will be able to read and execute it.

Note: You’ll want to write down the passphrase and store it in a safe location (preferably in two separate locations). That way, in case you need to restore the backups, you won’t have useless encrypted files.

Now edit your crontab and add a line like the following:

10 1 * * 0 /path/to/duplicity-wrapper.py /path/to/folder/ s3+http://bucket-name/somefolder &>> ~/log/backups.log

This will create a weekly backup for /path/to/folder. The backup will be encrypted with whatever passphrase you’ve given in the duplicity-wrapper.py. The output of the backup process will be saved in ~/log/backups.log.

You should also run

/path/to/duplicity-wrapper.py full /path/to/folder/ s3+http://bucket-name/somefolder

in order to create full backups. You might want to periodically verify your backups:

/path/to/duplicity-wrapper.py collection-status s3+http://bucket-name/somefolder
/path/to/duplicity-wrapper.py verify s3+http://bucket-name/somefolder /path/to/folder/

to check the status of the backups and verify them.

And last but not least, in case you ever need the backups, you can restore them using:

/path/to/duplicity-wrapper.py restore s3+http://bucket-name/somefolder /path/to/folder/

Security Considerations

As I know some people will comment on saving the encryption passphrase plainly in a file, I will explain my reasoning. I use the above encryption in order to secure my files in case of data leakage from Amazon S3. In order to read my backups, or silently tamper with them, someone will have to get the passphrase from my machine. While this isn’t impossible, I will say it’s unlikely. Furthermore, if someone has access that allows him to read files from my computer, he doesn’t need the backups; he can access the files directly.

I’ve given some thought to making the backups more secure, but it seems you always have to compromise on either automation or incremental backups. But, as I wrote, the current solution seems to me strong enough given the circumstances. Nonetheless, if you’ve got a better solution, it would be nice to hear.