<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Guy Rutenberg &#187; backup</title>
	<atom:link href="http://www.guyrutenberg.com/tag/backup/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.guyrutenberg.com</link>
	<description>Keeping track of what I do</description>
	<lastBuildDate>Sat, 14 Jan 2012 11:30:25 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Gmail backup: getmail vs. OfflineIMAP</title>
		<link>http://www.guyrutenberg.com/2012/01/14/gmail-backup-getmail-vs-offlineimap/</link>
		<comments>http://www.guyrutenberg.com/2012/01/14/gmail-backup-getmail-vs-offlineimap/#comments</comments>
		<pubDate>Sat, 14 Jan 2012 11:30:25 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Tips]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[duplicity]]></category>
		<category><![CDATA[getmail]]></category>
		<category><![CDATA[Google]]></category>
		<category><![CDATA[OfflineIMAP]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=956</guid>
		<description><![CDATA[I&#8217;m currently reviewing my backup plans and decided it&#8217;s a good occasion to finally start backing up my Gmail account. Firstly, I didn&#8217;t seriously consider Desktop clients as the main backup tool, as they are hard to automate. The two main options are: OfflineIMAP and getamil. Both are available from Ubuntu&#8217;s repositories, so installation is [...]]]></description>
			<content:encoded><![CDATA[<p>I&#8217;m currently reviewing my backup plans and decided it&#8217;s a good occasion to finally start backing up my Gmail account. Firstly, I didn&#8217;t seriously consider Desktop clients as the main backup tool, as they are hard to automate. The two main options are: <a href="http://offlineimap.org/">OfflineIMAP</a> and <a href="http://pyropus.ca/software/getmail/">getamil</a>. Both are available from Ubuntu&#8217;s repositories, so installation is easy with both and both have good tutorials, <a href="http://www.mattcutts.com/blog/backup-gmail-in-linux-with-getmail/">Matt Cutts&#8217; getmail</a> and <a href="http://www.enigmacurry.com/2008/02/22/backing-up-my-online-brain/">EnigmaCurry&#8217;s OfflineIMAP</a>.</p>
<p>OfflineIMAP claims to be faster, but I haven&#8217;t really checked it (and I&#8217;m not sure how important that is giving that it runs in the background). From what I saw configuring them is mainly a task of cut-and-paste, but getmail requires to list every label you want to backup, which I consider is a major downside. As both are able to save the mails to maildir format, it should be easy to back it up using <a href="duplicity.nongnu.org">duplicity</a>.</p>
<p>Conclusion: This was a short comparison, mainly to guide me in choosing the right backup for me, you may have different opinions (which, of course, I would gladly hear). I finally chose OfflineIMAP, mainly due to the labels issue.</p>
<p>Note on desktop clients: It seems that every decent one can be configured to work with a local maildir, so you can use them to read the backups. As I prefer Gmail&#8217;s interface, I will only use desktop clients in case I&#8217;m offline, so read-only access from desktop client seems good enough for me.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2012/01/14/gmail-backup-getmail-vs-offlineimap/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Automated Encrypted Backups to S3 Using Duplicity</title>
		<link>http://www.guyrutenberg.com/2011/09/03/automated-encrypted-backups-to-s3-using-duplicity/</link>
		<comments>http://www.guyrutenberg.com/2011/09/03/automated-encrypted-backups-to-s3-using-duplicity/#comments</comments>
		<pubDate>Sat, 03 Sep 2011 17:28:32 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Tips]]></category>
		<category><![CDATA[Amazon AWS]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[duplicity]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=864</guid>
		<description><![CDATA[This tutorial will hopefully guide you in making automated encrypted backups to Amazon&#8217;s S3 using duplicity. It was written as a followup for Using Duplicity and Amazon S3 – Notes and Examples, in order to organize all the necessary information into a simple tutorial. Will start by creating a simple wrapper to duplicity: #! /usr/bin/python [...]]]></description>
			<content:encoded><![CDATA[<p>This tutorial will hopefully guide you in making automated encrypted backups to Amazon&#8217;s S3 using duplicity. It was written as a followup for <a href="/2009/12/12/using-duplicity-and-amazon-s3-notes-and-examples/">Using Duplicity and Amazon S3 – Notes and Examples</a>, in order to organize all the necessary information into a simple tutorial.</p>
<p>Will start by creating a simple wrapper to <code>duplicity</code>:</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#! /usr/bin/python</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">sys</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span>
&nbsp;
duplicity_bin = <span style="color: #483d8b;">'/usr/bin/duplicity'</span>
&nbsp;
env = <span style="color: black;">&#123;</span>
    <span style="color: #483d8b;">'AWS_ACCESS_KEY_ID'</span>:     <span style="color: #483d8b;">'PUT YOUR KEY ID HERE'</span>,
    <span style="color: #483d8b;">'AWS_SECRET_ACCESS_KEY'</span>: <span style="color: #483d8b;">'PUT YOUR SECRET ACCESS KEY HERE'</span>,
    <span style="color: #483d8b;">'PASSPHRASE'</span>:            <span style="color: #483d8b;">'PUT ENCRYPTION PASSPHRASE'</span>,
<span style="color: black;">&#125;</span>
env.<span style="color: black;">update</span><span style="color: black;">&#40;</span><span style="color: #dc143c;">os</span>.<span style="color: black;">environ</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #dc143c;">os</span>.<span style="color: black;">execve</span><span style="color: black;">&#40;</span>duplicity_bin, <span style="color: #dc143c;">sys</span>.<span style="color: black;">argv</span>, env<span style="color: black;">&#41;</span></pre></div></div>

<p>save this under <code>dupicity-wrapper.py</code> and <code>chmod 0500</code> it so only you will be able to read and execute it.</p>
<p><strong>Note:</strong> You&#8217;ll want to write down the passphrase and store it in a safe location (preferably in two separate locations). So in case you&#8217;ll need to restore the backups, you won&#8217;t have useless encrypted files.</p>
<p>Now edit your <code>crontab</code> and add a line like the following:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">10 1 * * 0 /path/to/duplicity-wrapper.py /path/to/folder/ s3+http://bucket-name/somefolder &amp;&gt;&gt; ~/log/backups.log</pre></div></div>

<p>This will create a weekly backup for <code>/path/to/folder</code>. The backup will be encrypted with what ever passphrase you&#8217;ve given in the <code>duplicity-wrapper.py</code>. The output of the backup process will be saved into <code>~/log/backups.log</code>.</p>
<p>You should also run</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">/path/to/duplicity-wrapper.py full /path/to/folder/ s3+http://bucket-name/somefolder</pre></div></div>

<p>in order to create full backups. You might want to periodically verify your backups:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">/path/to/duplicity-wrapper.py collection-status s3+http://bucket-name/somefolder
/path/to/duplicity-wrapper.py verify s3+http://bucket-name/somefolder /path/to/folder/</pre></div></div>

<p>To check the status of the backups and to verify them.</p>
<p>And last but not least, in case you ever need the backups, you can restore them using:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">/path/to/duplicity-wrapper.py restore s3+http://bucket-name/somefolder /path/to/folder/</pre></div></div>

<h3>Security Considerations</h3>
<p>As I know, some people will comment on the saving of the encryption passphrase plainly in a file, I will explain my reasoning. I use the above encryption in order to secure my files in case of data leakage from Amazon S3. In order to read my backups, or silently temper with them, some on will have to get the passphrase from my machine. While, this isn&#8217;t impossible, I will say it&#8217;s unlikely. Furthermore, if someone has access allowing him to read files from my computer, he doesn&#8217;t need the backups, he can access the files directly.</p>
<p>I&#8217;ve given some thought about making the backups more secure, but it seems you always have to compromise on either automation or incremental backups. But, as I wrote, the current solution seems to me strong enough given the circumstances. Nonetheless, if you&#8217;ve got a better solution it would be nice to hear.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2011/09/03/automated-encrypted-backups-to-s3-using-duplicity/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Django Backup Script</title>
		<link>http://www.guyrutenberg.com/2010/10/14/django-backup-script/</link>
		<comments>http://www.guyrutenberg.com/2010/10/14/django-backup-script/#comments</comments>
		<pubDate>Thu, 14 Oct 2010 20:01:42 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[Django]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=748</guid>
		<description><![CDATA[This is a backup script for Django projects. It&#8217;s able to automate backups to a local folder and a remote FTP server of both the database and files. It somewhat old and has a few limitations, mainly supporting only MySQL and not supporting the new way for specifying databases introduced in Django 1.2. It&#8217;s loosly [...]]]></description>
			<content:encoded><![CDATA[<p>This is a backup script for Django projects. It&#8217;s able to automate backups to a local folder and a remote FTP server of both the database and files. It somewhat old and has a few limitations, mainly supporting only MySQL and not supporting the new way for specifying databases introduced in Django 1.2.</p>
<p>It&#8217;s loosly based on my <a href="/2009/01/06/wordpress-backup-to-ftp/">WordPress backup script</a> and inspired the database settings auto-detection found in the <a href="/2010/02/28/improved-ftp-backup-for-wordpress/">newer Wordrpess backup script</a>.</p>
<p>Usage is simple:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ django_backup /path/to/my/proj
$ django_backup --db-only /path/to/my/proj</pre></div></div>

<p>The latter command only backups the database.</p>
<p>The script uses a few configuration variables in the top of the script to set the folder which the local backups are kept in and the remote FTP server settings. The database settings are extracted directly from the <code>settings.py</code> of the backed-up project.<br />
<span id="more-748"></span></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># (C) 2009 Guy Rutenberg</span>
<span style="color: #666666; font-style: italic;"># Backup Django sites</span>
&nbsp;
<span style="color: #007800;">BACKUP_DIR</span>=
<span style="color: #007800;">FTP_HOST</span>=
<span style="color: #007800;">FTP_USER</span>=
<span style="color: #007800;">FTP_PASS</span>=
<span style="color: #007800;">FTP_BACKUP_DIR</span>=
&nbsp;
<span style="color: #666666; font-style: italic;"># end of user configurable section</span>
&nbsp;
<span style="color: #007800;">PROG</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;$0&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
print_usage <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #7a0874; font-weight: bold;">&#123;</span>
    <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;USAGE: $0 [options] PROJ_ROOT&quot;</span>
    <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Backup a Django project located in PROJ_ROOT&quot;</span>
<span style="color: #7a0874; font-weight: bold;">&#125;</span>
&nbsp;
print_help <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>  <span style="color: #7a0874; font-weight: bold;">&#123;</span>
    print_usage
    <span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt; EOF
&nbsp;
Options:
    -h, --help          show this help message and exit
    --db-only           only backup the database
&nbsp;
EOF</span>
<span style="color: #7a0874; font-weight: bold;">&#125;</span>
&nbsp;
<span style="color: #007800;">TEMP</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">getopt</span> <span style="color: #660033;">-o</span> h <span style="color: #660033;">--long</span> <span style="color: #7a0874; font-weight: bold;">help</span>,db-only <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$PROG</span>&quot;</span> <span style="color: #660033;">--</span> <span style="color: #ff0000;">&quot;$@&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #007800;">$?</span><span style="color: #7a0874; font-weight: bold;">&#41;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>; <span style="color: #000000; font-weight: bold;">then</span>
    print_usage
    <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">eval</span> <span style="color: #000000; font-weight: bold;">set</span> <span style="color: #660033;">--</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$TEMP</span>&quot;</span>
&nbsp;
<span style="color: #007800;">DB_ONLY</span>=<span style="color: #000000;">0</span>
<span style="color: #000000; font-weight: bold;">while</span> <span style="color: #c20cb9; font-weight: bold;">true</span> ; <span style="color: #000000; font-weight: bold;">do</span>
    <span style="color: #000000; font-weight: bold;">case</span> <span style="color: #ff0000;">&quot;$1&quot;</span> <span style="color: #000000; font-weight: bold;">in</span>
        <span style="color: #660033;">-h</span><span style="color: #000000; font-weight: bold;">|</span>--help<span style="color: #7a0874; font-weight: bold;">&#41;</span> print_help; <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000; font-weight: bold;">;;</span>
        --db-only<span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #007800;">DB_ONLY</span>=<span style="color: #ff0000;">&quot;1&quot;</span> ; <span style="color: #7a0874; font-weight: bold;">shift</span> <span style="color: #000000; font-weight: bold;">;;</span>
        --<span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #7a0874; font-weight: bold;">shift</span>; <span style="color: #7a0874; font-weight: bold;">break</span><span style="color: #000000; font-weight: bold;">;;</span>
    <span style="color: #000000; font-weight: bold;">esac</span>
<span style="color: #000000; font-weight: bold;">done</span>
&nbsp;
&nbsp;
&nbsp;
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-z</span> <span style="color: #ff0000;">&quot;$1&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>
<span style="color: #000000; font-weight: bold;">then</span>
 print_usage <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span>stderr
 <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #007800;">PROJECT_DIR</span>=<span style="color: #007800;">$1</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># extract database variables from settings.py</span>
<span style="color: #7a0874; font-weight: bold;">cd</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$PROJECT_DIR</span>&quot;</span>
&nbsp;
&nbsp;
<span style="color: #007800;">DB_ENGINE</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_ENGINE&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DB_ENGINE</span>&quot;</span> <span style="color: #000000; font-weight: bold;">!</span>= <span style="color: #ff0000;">&quot;mysql&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
 <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #007800;">$DB_ENGINE</span>
 <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Only mysql databases are supported!&quot;</span><span style="color: #000000; font-weight: bold;">&gt;/</span>dev<span style="color: #000000; font-weight: bold;">/</span>stderr
 <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #007800;">DB_NAME</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_NAME&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_USER</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_USER&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_PASS</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_PASSWORD&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_HOST</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_HOST&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #666666; font-style: italic;">#TODO find how to use it</span>
<span style="color: #007800;">DB_PORT</span>=<span style="color: #000000; font-weight: bold;">`</span>python <span style="color: #660033;">-c</span> <span style="color: #ff0000;">&quot;from settings import *; print DATABASE_PORT&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># set optional parameters: host, port</span>
<span style="color: #007800;">HOST_ARGS</span>=<span style="color: #ff0000;">''</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DB_HOST</span>&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
 <span style="color: #007800;">HOST_ARGS</span>=<span style="color: #ff0000;">&quot;--host <span style="color: #007800;">$DB_HOST</span>&quot;</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #007800;">PORT_ARGS</span>=<span style="color: #ff0000;">''</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DB_PORT</span>&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
 <span style="color: #007800;">PORT_ARGS</span>=<span style="color: #ff0000;">&quot;--port <span style="color: #007800;">$DB_HOST</span>&quot;</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #007800;">SITE_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$PROJECT_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$PROJECT_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">BACKUP_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;dumping database... &quot;</span>
<span style="color: #007800;">DUMP_NAME</span>=<span style="color: #800000;">${DB_NAME}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.sql.bz2
mysqldump <span style="color: #660033;">--user</span>=<span style="color: #800000;">${DB_USER}</span> <span style="color: #660033;">--password</span>=<span style="color: #800000;">${DB_PASS}</span> <span style="color: #007800;">$HOST_ARGS</span> \
 <span style="color: #007800;">$PORT_ARGS</span> <span style="color: #660033;">--databases</span> <span style="color: #800000;">${DB_NAME}</span> \
 <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">bzip2</span> <span style="color: #660033;">-c</span> <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${DUMP_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #007800;">$?</span><span style="color: #7a0874; font-weight: bold;">&#41;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #007800;">PUT_TARBALL_FTP</span>=<span style="color: #ff0000;">&quot;&quot;</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$DB_ONLY</span>&quot;</span> <span style="color: #660033;">-eq</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Creating tarball... &quot;</span>
	<span style="color: #007800;">TAR_NAME</span>=<span style="color: #800000;">${SITE_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2
	<span style="color: #c20cb9; font-weight: bold;">tar</span> <span style="color: #660033;">-cjf</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${SITE_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2 <span style="color: #800000;">${SITE_DIR}</span>
	<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #007800;">$?</span><span style="color: #7a0874; font-weight: bold;">&#41;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>; <span style="color: #000000; font-weight: bold;">then</span>
		<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
		<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">2</span>
	<span style="color: #000000; font-weight: bold;">fi</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
	<span style="color: #007800;">PUT_TARBALL_FTP</span>=<span style="color: #ff0000;">&quot;put <span style="color: #000099; font-weight: bold;">\&quot;</span><span style="color: #007800;">${BACKUP_DIR}</span>/<span style="color: #007800;">${TAR_NAME}</span><span style="color: #000099; font-weight: bold;">\&quot;</span>&quot;</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Uploading backup to FTP... &quot;</span>
&nbsp;
lftp <span style="color: #660033;">-u</span> <span style="color: #800000;">${FTP_USER}</span>,<span style="color: #800000;">${FTP_PASS}</span> <span style="color: #800000;">${FTP_HOST}</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;EOF
cd &quot;${FTP_BACKUP_DIR}&quot;
put &quot;${BACKUP_DIR}/${DUMP_NAME}&quot;
${PUT_TARBALL_FTP}
&nbsp;
EOF</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #007800;">$?</span><span style="color: #7a0874; font-weight: bold;">&#41;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">3</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2010/10/14/django-backup-script/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Improved FTP Backup for WordPress</title>
		<link>http://www.guyrutenberg.com/2010/02/28/improved-ftp-backup-for-wordpress/</link>
		<comments>http://www.guyrutenberg.com/2010/02/28/improved-ftp-backup-for-wordpress/#comments</comments>
		<pubDate>Sun, 28 Feb 2010 06:38:44 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[Projects]]></category>
		<category><![CDATA[Wordpress]]></category>
		<category><![CDATA[backup]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=637</guid>
		<description><![CDATA[This script backups both the database and files of a WordPress blog into a remote FTP server (while keeping a local copy). It&#8217;s an update of my WordPress Backup to FTP script. The main changes are auto-detecting database settings and better support for caching plugins (specifically WP-Cache). The new version makes it easier to backup [...]]]></description>
			<content:encoded><![CDATA[<p>This script backups both the database and files of a WordPress blog into a remote FTP server (while keeping a local copy). It&#8217;s an update of my <a href="/2009/01/06/wordpress-backup-to-ftp/">WordPress Backup to FTP</a> script. The main changes are auto-detecting database settings and better support for caching plugins (specifically WP-Cache). The new version makes it easier to backup multiple WordPress blogs to the same FTP server.<br />
<span id="more-637"></span><br />
Usage is pretty simple after a short initial configuration. First, save the the script and make it executable:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #c20cb9; font-weight: bold;">chmod</span> +x wp-backup</pre></div></div>

<p>(assuming you saved it under the name <code>wp-backup</code>). After saving it edit the file with your favorite editor and set the 5 configuration variable to whatever is appropriate for you. <code>BACKUP_DIR</code> is the folder to save the local backups to. <code>FTP_HOST</code>, <code>FTP_USER</code>, <code>FTP_PASS</code> control the FTP host, username and password, respectively, for the remote backup server. <code>FTP_BACKUP_DIR</code> sets the folder on the FTP server to save the remote backup to.</p>
<p>Now that the initial configuration is done, all you need to do is execute the script and give the path to the blog as an argument. For example:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">.<span style="color: #000000; font-weight: bold;">/</span>wp-backup <span style="color: #000000; font-weight: bold;">/</span>home<span style="color: #000000; font-weight: bold;">/</span>someuser<span style="color: #000000; font-weight: bold;">/</span>myblog</pre></div></div>

<p>And that it, the script will backup your files (excluding cache) and database to both a local and remote locations. This allows using the same script to backup multiple WordPress blogs, unlike the previous script which had to be modified for each blog.</p>
<p>And now the script itself:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Copyright 2008, 2010 Guy Rutenberg &lt;http://www.guyrutenberg.com/contact-me&gt;</span>
<span style="color: #666666; font-style: italic;"># WordPress FTP backup 2.0</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># Easily backup wordpress instances via ftp.</span>
<span style="color: #666666; font-style: italic;">#</span>
<span style="color: #666666; font-style: italic;"># Change Log:</span>
<span style="color: #666666; font-style: italic;"># ===========</span>
<span style="color: #666666; font-style: italic;"># 2.0:</span>
<span style="color: #666666; font-style: italic;">#  - Auto-detect database settings.</span>
<span style="color: #666666; font-style: italic;">#  - Exclude cache data from backups.</span>
&nbsp;
<span style="color: #007800;">BACKUP_DIR</span>=
<span style="color: #007800;">FTP_HOST</span>=
<span style="color: #007800;">FTP_USER</span>=
<span style="color: #007800;">FTP_PASS</span>=
<span style="color: #007800;">FTP_BACKUP_DIR</span>=
&nbsp;
<span style="color: #666666; font-style: italic;"># end of configuration - you probably don't need to touch anything bellow</span>
&nbsp;
<span style="color: #007800;">PROG</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;$0&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
print_usage <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #7a0874; font-weight: bold;">&#123;</span>
    <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;USAGE: <span style="color: #007800;">${PROG}</span> [options] BLOG_ROOT&quot;</span>
    <span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;Backup a WordPress blog&quot;</span>
<span style="color: #7a0874; font-weight: bold;">&#125;</span>
&nbsp;
print_help <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>  <span style="color: #7a0874; font-weight: bold;">&#123;</span>
    print_usage
    <span style="color: #c20cb9; font-weight: bold;">cat</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt; EOF
&nbsp;
Options:
    -h, --help          show this help message and exit
&nbsp;
EOF</span>
<span style="color: #7a0874; font-weight: bold;">&#125;</span>
&nbsp;
<span style="color: #007800;">TEMP</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">getopt</span> <span style="color: #660033;">-o</span> h <span style="color: #660033;">--long</span> <span style="color: #7a0874; font-weight: bold;">help</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$PROG</span>&quot;</span> <span style="color: #660033;">--</span> <span style="color: #ff0000;">&quot;$@&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #007800;">$?</span><span style="color: #7a0874; font-weight: bold;">&#41;</span><span style="color: #7a0874; font-weight: bold;">&#41;</span>; <span style="color: #000000; font-weight: bold;">then</span>
    print_usage
    <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">eval</span> <span style="color: #000000; font-weight: bold;">set</span> <span style="color: #660033;">--</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$TEMP</span>&quot;</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">while</span> <span style="color: #c20cb9; font-weight: bold;">true</span> ; <span style="color: #000000; font-weight: bold;">do</span>
    <span style="color: #000000; font-weight: bold;">case</span> <span style="color: #ff0000;">&quot;$1&quot;</span> <span style="color: #000000; font-weight: bold;">in</span>
        <span style="color: #660033;">-h</span><span style="color: #000000; font-weight: bold;">|</span>--help<span style="color: #7a0874; font-weight: bold;">&#41;</span> print_help; <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000; font-weight: bold;">;;</span>
        --<span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #7a0874; font-weight: bold;">shift</span>; <span style="color: #7a0874; font-weight: bold;">break</span><span style="color: #000000; font-weight: bold;">;;</span>
    <span style="color: #000000; font-weight: bold;">esac</span>
<span style="color: #000000; font-weight: bold;">done</span>
&nbsp;
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #660033;">-z</span> <span style="color: #ff0000;">&quot;$1&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>
<span style="color: #000000; font-weight: bold;">then</span>
 print_usage <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #000000; font-weight: bold;">/</span>dev<span style="color: #000000; font-weight: bold;">/</span>stderr
 <span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
&nbsp;
<span style="color: #007800;">BLOG_DIR</span>=<span style="color: #007800;">$1</span>
<span style="color: #007800;">DB_NAME</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;&lt;?php require_once(<span style="color: #000099; font-weight: bold;">\&quot;</span><span style="color: #007800;">${BLOG_DIR}</span>/wp-config.php<span style="color: #000099; font-weight: bold;">\&quot;</span>); echo DB_NAME;&quot;</span> <span style="color: #000000; font-weight: bold;">|</span> php<span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_USER</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;&lt;?php require_once(<span style="color: #000099; font-weight: bold;">\&quot;</span><span style="color: #007800;">${BLOG_DIR}</span>/wp-config.php<span style="color: #000099; font-weight: bold;">\&quot;</span>); echo DB_USER;&quot;</span> <span style="color: #000000; font-weight: bold;">|</span> php<span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_PASS</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;&lt;?php require_once(<span style="color: #000099; font-weight: bold;">\&quot;</span><span style="color: #007800;">${BLOG_DIR}</span>/wp-config.php<span style="color: #000099; font-weight: bold;">\&quot;</span>); echo DB_PASSWORD;&quot;</span> <span style="color: #000000; font-weight: bold;">|</span> php<span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">DB_HOST</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;&lt;?php require_once(<span style="color: #000099; font-weight: bold;">\&quot;</span><span style="color: #007800;">${BLOG_DIR}</span>/wp-config.php<span style="color: #000099; font-weight: bold;">\&quot;</span>); echo DB_HOST;&quot;</span> <span style="color: #000000; font-weight: bold;">|</span> php<span style="color: #000000; font-weight: bold;">`</span>
&nbsp;
<span style="color: #007800;">BLOG_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BLOG_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BLOG_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">BACKUP_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;dumping database... &quot;</span>
<span style="color: #007800;">DUMP_NAME</span>=<span style="color: #800000;">${DB_NAME}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.sql.bz2
mysqldump <span style="color: #660033;">--user</span>=<span style="color: #800000;">${DB_USER}</span> <span style="color: #660033;">--password</span>=<span style="color: #800000;">${DB_PASS}</span> <span style="color: #660033;">--host</span>=<span style="color: #800000;">${DB_HOST}</span> \
 <span style="color: #660033;">--databases</span> <span style="color: #800000;">${DB_NAME}</span> \
 <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">bzip2</span> <span style="color: #660033;">-c</span> <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${DUMP_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Creating tarball... &quot;</span>
<span style="color: #007800;">TAR_NAME</span>=<span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2
<span style="color: #c20cb9; font-weight: bold;">tar</span> <span style="color: #660033;">-cjf</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2 <span style="color: #660033;">--exclude</span> cache <span style="color: #800000;">${BLOG_DIR}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">2</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Uploading SQL dump and tarball to FTP... &quot;</span>
lftp <span style="color: #660033;">-u</span> <span style="color: #800000;">${FTP_USER}</span>,<span style="color: #800000;">${FTP_PASS}</span> <span style="color: #800000;">${FTP_HOST}</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;EOF
cd &quot;${FTP_BACKUP_DIR}&quot;
put &quot;${BACKUP_DIR}/${DUMP_NAME}&quot;
put &quot;${BACKUP_DIR}/${TAR_NAME}&quot;
&nbsp;
EOF</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">3</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2010/02/28/improved-ftp-backup-for-wordpress/feed/</wfw:commentRss>
		<slash:comments>43</slash:comments>
		</item>
		<item>
		<title>Using Duplicity and Amazon S3 &#8211; Notes and Examples</title>
		<link>http://www.guyrutenberg.com/2009/12/12/using-duplicity-and-amazon-s3-notes-and-examples/</link>
		<comments>http://www.guyrutenberg.com/2009/12/12/using-duplicity-and-amazon-s3-notes-and-examples/#comments</comments>
		<pubDate>Sat, 12 Dec 2009 10:00:14 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Tips]]></category>
		<category><![CDATA[Amazon AWS]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[duplicity]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=603</guid>
		<description><![CDATA[Up until now I&#8217;ve been doing my backups to Amazon S3 using my s3backup script. While it&#8217;s simple and does what I needed at the time, I&#8217;ve decided to cut some of the costs by switching to incremental backups. I went on to define what I&#8217;m looking for, and after a short search I&#8217;ve came [...]]]></description>
			<content:encoded><![CDATA[<p>Up until now I&#8217;ve been doing my backups to Amazon S3 using my <a href="/2008/10/21/s3backup-easy-backups-of-folders-to-amazon-s3/"><code>s3backup</code></a> script. While it&#8217;s simple and does what I needed at the time, I&#8217;ve decided to cut some of the costs by switching to incremental backups.<br />
<span id="more-603"></span><br />
I went on to define what I&#8217;m looking for, and after a short search I&#8217;ve came up with <a href="http://duplicity.nongnu.org/">duplicity</a> which support efficient incremental backups to Amazon S3 (among many other backends). While duplicity has an simple CLI interface, I did come across two pitfalls when using S3.</p>
<p>The first one is that one must export the <code>AWS_ACCESS_KEY_ID</code> and <code>AWS_SECRET_ACCESS_KEY</code> or else you get a error message from the underlying <code>boto</code> library:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">  File &quot;//usr/lib64/python2.5/site-packages/boto/connection.py&quot;, line 148, in __init__
    self.hmac = hmac.new(self.aws_secret_access_key, digestmod=sha)
AttributeError: S3Connection instance has no attribute 'aws_secret_access_key'</pre></div></div>

<p>The second thing to note is that is the way to specify buckets for <code>duplicity</code>. Instead of</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">s3://&lt;bucket-name&gt;/&lt;prefix&gt;</pre></div></div>

<p>which is used by <a href="http://s3tools.logix.cz/s3cmd"><code>s3cmd</code></a> (which is a great tool), it asks to specify it using</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">s3+http://&lt;bucket-name&gt;/&lt;prefix&gt;</pre></div></div>

<p>Being aware of these points can save some time and frustration. In order to automate the backup process one can use <code>cron</code>. For example add the following to your <code>crontab</code></p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">AWS_ACCESS_KEY_ID=&quot;&lt;your-key-id&gt;&quot;
AWS_SECRET_ACCESS_KEY=&quot;&lt;your-secret-access-key&gt;&quot;
0 1 * * 0  duplicity --no-encryption &lt;folder-to-backup&gt; s3+http://&lt;bucket-name&gt;/&lt;prefix&gt; &amp;&gt;&gt; ~/backups.log
AWS_ACCESS_KEY_ID=&quot;&quot;
AWS_SECRET_ACCESS_KEY=&quot;&quot;</pre></div></div>

<p>I unset the environment variables, so it won&#8217;t leak to other cron jobs unnecessarily. </p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2009/12/12/using-duplicity-and-amazon-s3-notes-and-examples/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>WordPress Backup to FTP</title>
		<link>http://www.guyrutenberg.com/2009/01/06/wordpress-backup-to-ftp/</link>
		<comments>http://www.guyrutenberg.com/2009/01/06/wordpress-backup-to-ftp/#comments</comments>
		<pubDate>Tue, 06 Jan 2009 13:46:33 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[Wordpress]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[FTP]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=187</guid>
		<description><![CDATA[Update: A newer version of the script is available. This script allows you to easily backup your WordPress blog to an FTP server. It&#8217;s actually a modification of my WordPress Backup to Amazon S3 Script, but instead of saving the backup to Amazon S3 it uploads it to an FTP server. Another update is that [...]]]></description>
			<content:encoded><![CDATA[<p><strong>Update:</strong> A newer version of the script is <a href="/2010/02/28/improved-ftp-backup-for-wordpress/">available</a>.</p>
<p>This script allows you to easily backup your WordPress blog to an FTP server. It&#8217;s actually a modification of my <a href="/2008/10/11/wordpress-backup-to-amazon-s3-script/">WordPress Backup to Amazon S3 Script</a>, but instead of saving the backup to Amazon S3 it uploads it to an FTP server. Another update is that now the SQL dump includes the database creation instructions so you don&#8217;t need to create it manually before restoring from the backup.</p>
<p>Although I&#8217;ve written it with WordPress in mind (to creates backups of my blog), it isn&#8217;t WordPress specific. It can be used to backup any website that consists of a MySQL database and files. I&#8217;ve successfully used it to backup MediaWiki installation.<br />
<span id="more-187"></span></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># (C) 2008 Guy Rutenberg</span>
<span style="color: #666666; font-style: italic;"># This is a script that creates backups of my blog.</span>
&nbsp;
<span style="color: #007800;">DB_NAME</span>=
<span style="color: #007800;">DB_USER</span>=
<span style="color: #007800;">DB_PASS</span>=
<span style="color: #007800;">DB_HOST</span>=
&nbsp;
<span style="color: #007800;">BLOG_DIR</span>=
<span style="color: #007800;">BACKUP_DIR</span>=
&nbsp;
<span style="color: #007800;">FTP_HOST</span>=
<span style="color: #007800;">FTP_USER</span>=
<span style="color: #007800;">FTP_PASS</span>=
<span style="color: #007800;">FTP_BACKUP_DIR</span>=
&nbsp;
<span style="color: #666666; font-style: italic;"># end of configuration - you probably don't need to touch anything bellow</span>
<span style="color: #007800;">BLOG_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BLOG_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BLOG_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
<span style="color: #007800;">BACKUP_DIR</span>=<span style="color: #000000; font-weight: bold;">`</span><span style="color: #c20cb9; font-weight: bold;">dirname</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`/`</span><span style="color: #c20cb9; font-weight: bold;">basename</span> <span style="color: #ff0000;">&quot;<span style="color: #007800;">$BACKUP_DIR</span>&quot;</span><span style="color: #000000; font-weight: bold;">`</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;dumping database... &quot;</span>
<span style="color: #007800;">DUMP_NAME</span>=<span style="color: #800000;">${DB_NAME}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.sql.bz2
mysqldump <span style="color: #660033;">--user</span>=<span style="color: #800000;">${DB_USER}</span> <span style="color: #660033;">--password</span>=<span style="color: #800000;">${DB_PASS}</span> <span style="color: #660033;">--host</span>=<span style="color: #800000;">${DB_HOST}</span> \
 <span style="color: #660033;">--databases</span> <span style="color: #800000;">${DB_NAME}</span> \
 <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">bzip2</span> <span style="color: #660033;">-c</span> <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${DUMP_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Creating tarball... &quot;</span>
<span style="color: #007800;">TAR_NAME</span>=<span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2
<span style="color: #c20cb9; font-weight: bold;">tar</span> <span style="color: #660033;">-cjf</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2 <span style="color: #800000;">${BLOG_DIR}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">2</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Uploading SQL dump and tarball to FTP... &quot;</span>
lftp <span style="color: #660033;">-u</span> <span style="color: #800000;">${FTP_USER}</span>,<span style="color: #800000;">${FTP_PASS}</span> <span style="color: #800000;">${FTP_HOST}</span> <span style="color: #cc0000; font-style: italic;">&lt;&lt;EOF
cd &quot;${FTP_BACKUP_DIR}&quot;
put &quot;${BACKUP_DIR}/${DUMP_NAME}&quot;
put &quot;${BACKUP_DIR}/${TAR_NAME}&quot;
&nbsp;
EOF</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">3</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span></pre></div></div>

<p>The configuration is done using the variables in the beginning of the script (until the &#8220;end of configuration&#8221; line). They are pretty self explanatory, and you should adjust them for your site before using the script. If there are variables you don&#8217;t understand what they do, see the post about the <a href="/2008/10/11/wordpress-backup-to-amazon-s3-script">previous script</a>, and don&#8217;t hesitate to comment.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2009/01/06/wordpress-backup-to-ftp/feed/</wfw:commentRss>
		<slash:comments>27</slash:comments>
		</item>
		<item>
		<title>Backup a SourceForge hosted SVN repository &#8211; sf-svn-backup</title>
		<link>http://www.guyrutenberg.com/2008/12/12/backup-a-sourceforge-hosted-svn-repository-sf-svn-backup/</link>
		<comments>http://www.guyrutenberg.com/2008/12/12/backup-a-sourceforge-hosted-svn-repository-sf-svn-backup/#comments</comments>
		<pubDate>Fri, 12 Dec 2008 17:49:30 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[svn]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=175</guid>
		<description><![CDATA[SourceForge urges their users to backup the code repositories of their projects. As I have several projects hosted with SourceForge, I should do it too. Making the backups isn&#8217;t complicated at all, but because it isn&#8217;t automated properly, I&#8217;ve been lazy with it. sf-svn-backup was written in order to simply automate the process. The script [...]]]></description>
			<content:encoded><![CDATA[<p>SourceForge urges their users to backup the code repositories of their projects. As I have several projects hosted with SourceForge, I should do it too. Making the backups isn&#8217;t complicated at all, but because it isn&#8217;t automated properly, I&#8217;ve been lazy with it.</p>
<p><a href="/wp-content/uploads/2008/12/sf-svn-backup.gz"><code>sf-svn-backup</code></a> was written in order to simply automate the process. The script is pretty simple to use, just pass as the first argument the project name and the script will write down to <code>stdout</code> the dump file.</p>
<p>For example:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">sf-svn-backup openyahtzee &gt; openyahtzee.dump</pre></div></div>

<p>The project name should be it&#8217;s UNIX name (e.g. openyahtzee and not Open Yahtzee). Because the script writes the dump file directly to <code>stdout</code> it&#8217;s easy to pipe the output first through a compression program such as <code>gzip</code> to create compressed SVN dump files.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2008/12/12/backup-a-sourceforge-hosted-svn-repository-sf-svn-backup/feed/</wfw:commentRss>
		<slash:comments>6</slash:comments>
		</item>
		<item>
		<title>s3backup &#8211; Easy backups of Folders to Amazon S3</title>
		<link>http://www.guyrutenberg.com/2008/10/21/s3backup-easy-backups-of-folders-to-amazon-s3/</link>
		<comments>http://www.guyrutenberg.com/2008/10/21/s3backup-easy-backups-of-folders-to-amazon-s3/#comments</comments>
		<pubDate>Tue, 21 Oct 2008 19:24:49 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[Amazon AWS]]></category>
		<category><![CDATA[backup]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=142</guid>
		<description><![CDATA[This is an updated version of my previous backups script &#8211; Backup Directories to Amazon S3 Script. The new script works much better and safer. Unlike the old script, the new one creates the tarballs in a temporary file under /tmp, and allows more control over the backup process. #! /bin/bash &#160; # Copyright (C) [...]]]></description>
			<content:encoded><![CDATA[<p>This is an updated version of my previous backups script &#8211; <a href="/2008/03/01/backup-directories-to-amazon-s3-script/">Backup Directories to Amazon S3 Script</a>. The new script works much better and safer. Unlike the old script, the new one creates the tarballs in a temporary file under <code>/tmp</code>, and allows more control over the backup process.</p>
<p><span id="more-142"></span></p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">#! /bin/bash
&nbsp;
# Copyright (C) 2008 by Guy Rutenberg
&nbsp;
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the
# Free Software Foundation, Inc.,
# 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
&nbsp;
PROG=s3backup
DESCRIPTION=&quot;Backup directories to Amazon S3&quot;
SYNOPSIS=&quot;${PROG} [options] DIR BUCKET&quot;
VERSION=0.2
&nbsp;
print_usage () {
    echo &quot;Usage: ${SYNOPSIS}&quot;
    echo &quot;Save a backup of DIR to Amazon S3 bucket BUCKET&quot;
}
print_help ()  {
    print_usage
    cat &lt;&lt; EOF
&nbsp;
Options:
    -h, --help          show this help message and exit
    --version           show program's version number and exit
    -f, --full-path     use the path to dir in archive instead of only dir
    -v, --verbose       be verbose about progress
&nbsp;
Example:
    s3backup ~/Documents guyru-backup/
EOF
}
&nbsp;
print_version () {
    echo &quot;$PROG $VERSION&quot;
    echo &quot;Copyright (C) 2008 Guy Rutenberg &lt;http://www.guyrutenberg.com&gt;&quot;
}
&nbsp;
# print only if verbose
vecho () {
    if (( &quot;$VERBOSE&quot; &gt;= &quot;$1&quot; )) ; then
        echo &quot;$2&quot;
    fi
}
VERBOSE=0
TEMP=`getopt -o hvf --long help,version,verbose,full-path -n '${PROG}' -- &quot;$@&quot;`
if [ $? != 0 ] ; then
    print_usage
    exit 1
fi
&nbsp;
eval set -- &quot;$TEMP&quot;
&nbsp;
while true ; do
    case &quot;$1&quot; in
        -h|--help) print_help; exit ;;
        --version) print_version; exit ;;
        -f|--full-path) FULL_PATH=1; shift ;; 
        -v|--verbose) let VERBOSE++ ; shift ;;
        --) shift; break;;
    esac
done
&nbsp;
# no directory passed
if [ -z &quot;$1&quot; ]; then
    echo &quot;$PROG: missing DIR&quot;
    echo &quot;&quot;
    print_usage
    exit 1
fi
&nbsp;
# no bucket passed
if [ -z &quot;$2&quot; ]; then
    echo &quot;$PROG: missing BUCKET&quot;
    echo &quot;&quot;
    print_usage
    exit 2
fi
BUCKET=$2
&nbsp;
&nbsp;
DIR=$(basename &quot;$1&quot;)
DIR_PATH=$(dirname &quot;$1&quot;)
&nbsp;
DATE=$(date +%Y%m%d)
&nbsp;
&nbsp;
if [ -z &quot;$FULL_PATH&quot; ] ; then
    cd &quot;$DIR_PATH&quot;
    TAR_PATH=&quot;$DIR&quot;
else
    TAR_PATH=&quot;$1&quot;
fi
&nbsp;
vecho 1 &quot;Creating tarball&quot;
TARNAME=`mktemp`
TAR_OPTIONS=&quot;-c&quot;
if (( $VERBOSE &gt; 0 )); then
    TAR_OPTIONS+=&quot;v&quot;
fi
tar $TAR_OPTIONS &quot;$TAR_PATH&quot; | bzip2 -9 -c &gt; &quot;${TARNAME}&quot;
&nbsp;
vecho 1 &quot;Sending to Amazon S3&quot;
S3_OPTIONS=&quot;&quot;
if (( $VERBOSE &gt; 0 )); then
    S3_OPTIONS+=&quot;v&quot;
fi
s3cmd put $S3_OPTIONS &quot;$TARNAME&quot; &quot;s3://${BUCKET}$DIR-$DATE.tar.bz2&quot;
&nbsp;
vecho 1 &quot;Deleting intermediate files&quot;
rm &quot;$TARNAME&quot;
&nbsp;
# vim:sw=4:softtabstop=4:expandtab</pre></div></div>

<p>The script can also be downloaded from <a href="/wp-content/uploads/2008/10/s3backup.gz">s3backup.gz</a>.</p>
<p>Script usage is pretty straight forward, just pass the folder to backup as the first argument to the script, and an S3 bucket name, to which the backup will be saved as the second. You can also consult <code>s3backup --help</code>.</p>
<p>Note that you can add a prefix to your backups by adding it to the S3 bucket. For example</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">s3backup ~/Documents bucketname/prefix/</pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2008/10/21/s3backup-easy-backups-of-folders-to-amazon-s3/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>WordPress Backup to Amazon S3 Script</title>
		<link>http://www.guyrutenberg.com/2008/10/11/wordpress-backup-to-amazon-s3-script/</link>
		<comments>http://www.guyrutenberg.com/2008/10/11/wordpress-backup-to-amazon-s3-script/#comments</comments>
		<pubDate>Sat, 11 Oct 2008 09:19:22 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Bash]]></category>
		<category><![CDATA[Wordpress]]></category>
		<category><![CDATA[Amazon AWS]]></category>
		<category><![CDATA[backup]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=110</guid>
		<description><![CDATA[This is an updated version of my WordPress Backup Script. The new version basically does the same thing: backup up a wordpress blog (actually any site that consists of files and a MySQL database). The new thing about the script is that instead of only saving the backup locally, it also uploads it to Amazon [...]]]></description>
			<content:encoded><![CDATA[<p>This is an updated version of my <a href="/2008/05/07/wordpress-backup-script/">WordPress Backup Script</a>. The new version basically does the same thing: backup up a wordpress blog (actually any site that consists of files and a MySQL database). The new thing about the script is that instead of only saving the backup locally, it also uploads it to Amazon S3.</p>
<p><span id="more-110"></span></p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">#!/bin/bash</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># (C) 2008 Guy Rutenberg</span>
<span style="color: #666666; font-style: italic;"># This is a script that creates backups of my blog.</span>
&nbsp;
<span style="color: #007800;">DB_NAME</span>=database_name
<span style="color: #007800;">DB_USER</span>=database_user
<span style="color: #007800;">DB_PASS</span>=<span style="color: #000000; font-weight: bold;">****</span>
<span style="color: #007800;">DB_HOST</span>=db_host
&nbsp;
<span style="color: #666666; font-style: italic;">#no trailing slash</span>
<span style="color: #007800;">BLOG_DIR</span>=<span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>blog
<span style="color: #007800;">BACKUP_DIR</span>=<span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>backups
<span style="color: #007800;">S3BUCKET</span>=s3:<span style="color: #000000; font-weight: bold;">//</span>bucket-name<span style="color: #000000; font-weight: bold;">/</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># end of configuration - you probably don't need to touch anything bellow</span>
<span style="color: #007800;">DUMP_NAME</span>=<span style="color: #800000;">${DB_NAME}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.sql.bz2
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;dumping database... &quot;</span>
mysqldump <span style="color: #660033;">--user</span>=<span style="color: #800000;">${DB_USER}</span> <span style="color: #660033;">--password</span>=<span style="color: #800000;">${DB_PASS}</span> <span style="color: #660033;">--host</span>=<span style="color: #800000;">${DB_HOST}</span> <span style="color: #800000;">${DB_NAME}</span> \
 <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #c20cb9; font-weight: bold;">bzip2</span> <span style="color: #660033;">-c</span> <span style="color: #000000; font-weight: bold;">&gt;</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${DUMP_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #007800;">TAR_NAME</span>=<span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Creating tarball... &quot;</span>
<span style="color: #c20cb9; font-weight: bold;">tar</span> <span style="color: #660033;">-cjf</span> <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${BLOG_DIR##*/}</span>-$<span style="color: #7a0874; font-weight: bold;">&#40;</span><span style="color: #c20cb9; font-weight: bold;">date</span> +<span style="color: #000000; font-weight: bold;">%</span>Y<span style="color: #000000; font-weight: bold;">%</span>m<span style="color: #000000; font-weight: bold;">%</span>d<span style="color: #7a0874; font-weight: bold;">&#41;</span>.tar.bz2 <span style="color: #800000;">${BLOG_DIR}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Uploading SQL dump to Amazon S3... &quot;</span>
s3cmd put <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${DUMP_NAME}</span> <span style="color: #800000;">${S3BUCKET}</span><span style="color: #800000;">${DUMP_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span>
&nbsp;
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #660033;">-n</span> <span style="color: #ff0000;">&quot;Uploading tarball to Amazon S3... &quot;</span>
s3cmd put <span style="color: #800000;">${BACKUP_DIR}</span><span style="color: #000000; font-weight: bold;">/</span><span style="color: #800000;">${TAR_NAME}</span> <span style="color: #800000;">${S3BUCKET}</span><span style="color: #800000;">${TAR_NAME}</span>
<span style="color: #000000; font-weight: bold;">if</span> <span style="color: #7a0874; font-weight: bold;">&#91;</span> <span style="color: #ff0000;">&quot;$?&quot;</span> <span style="color: #660033;">-ne</span> <span style="color: #ff0000;">&quot;0&quot;</span> <span style="color: #7a0874; font-weight: bold;">&#93;</span>; <span style="color: #000000; font-weight: bold;">then</span>
	<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;failed!&quot;</span>
	<span style="color: #7a0874; font-weight: bold;">exit</span> <span style="color: #000000;">1</span>
<span style="color: #000000; font-weight: bold;">fi</span>
<span style="color: #7a0874; font-weight: bold;">echo</span> <span style="color: #ff0000;">&quot;done&quot;</span></pre></div></div>

<p>As with the older <a href="/2008/03/01/backup-directories-to-amazon-s3-script/">Amazon S3 backup script</a>, this one also required you to install the <a href="http://s3tools.logix.cz/s3cmd">s3cmd</a> package.</p>
<p>The first six variables:</p>
<ul>
<li><code>DB_NAME</code></li>
<li><code>DB_USER</code></li>
<li><code>DB_PASS</code></li>
<li><code>DB_HOST</code></li>
<li><code>BLOG_DIR</code></li>
<li><code>BACKUP_DIR</code></li>
</ul>
<p>Are the pretty self explanatory and have the same meaning as in the <a href="/2008/05/07/wordpress-backup-script/">old script</a>.</p>
<p>The script has one new variable: <code>S3BUCKET</code>. This should be set to the S3 bucket to which the backups should be saved. If you want to add some prefix to the backups files on S3 this is the place.</p>
<p>The script keeps the backups both in the local backup directory and the Amazon S3.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2008/10/11/wordpress-backup-to-amazon-s3-script/feed/</wfw:commentRss>
		<slash:comments>17</slash:comments>
		</item>
		<item>
		<title>Retrieving Google&#8217;s Cache for a Whole Website</title>
		<link>http://www.guyrutenberg.com/2008/10/02/retrieving-googles-cache-for-a-whole-website/</link>
		<comments>http://www.guyrutenberg.com/2008/10/02/retrieving-googles-cache-for-a-whole-website/#comments</comments>
		<pubDate>Thu, 02 Oct 2008 20:07:30 +0000</pubDate>
		<dc:creator>Guy</dc:creator>
				<category><![CDATA[Python]]></category>
		<category><![CDATA[backup]]></category>
		<category><![CDATA[Google]]></category>

		<guid isPermaLink="false">http://www.guyrutenberg.com/?p=92</guid>
		<description><![CDATA[Some time ago, as some of you noticed, the web server that hosts my blog went down. Unfortunately, some of the sites had no proper backup, so some thing had to be done in case the hard disk couldn&#8217;t be recovered. My efforts turned to Google&#8217;s cache. Google keeps a copy of the text of [...]]]></description>
			<content:encoded><![CDATA[<p>Some time ago, as some of you noticed, the web server that hosts my blog went down. Unfortunately, some of the sites had no proper backup, so some thing had to be done in case the hard disk couldn&#8217;t be recovered. My efforts turned to Google&#8217;s cache. Google keeps a copy of the text of the web page in it&#8217;s cache, something that is usually useful when the website is temporarily unavailable. The basic idea is to retrieve a copy of all the pages of a certain site that Google has a cache of.<br />
<span id="more-92"></span><br />
While this is easily done manually when only few pages are cached, the task needs to be automated when a need for retrieving several hundreds of pages rises. This is exactly what the following Python script does.</p>

<div class="wp_syntax"><div class="code"><pre class="python" style="font-family:monospace;"><span style="color: #808080; font-style: italic;">#!/usr/bin/python</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">urllib</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">urllib2</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">re</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">socket</span>
<span style="color: #ff7700;font-weight:bold;">import</span> <span style="color: #dc143c;">os</span>
<span style="color: #dc143c;">socket</span>.<span style="color: black;">setdefaulttimeout</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">30</span><span style="color: black;">&#41;</span>
<span style="color: #808080; font-style: italic;">#adjust the site here</span>
search_term=<span style="color: #483d8b;">&quot;site:guyrutenberg.com&quot;</span>
<span style="color: #ff7700;font-weight:bold;">def</span> main<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>:
    headers = <span style="color: black;">&#123;</span><span style="color: #483d8b;">'User-Agent'</span>: <span style="color: #483d8b;">'Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.4) Gecko/20070515 Firefox/2.0.0.4'</span><span style="color: black;">&#125;</span>
    url = <span style="color: #483d8b;">&quot;http://www.google.com/search?q=&quot;</span>+search_term
    regex_cache = <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span>r<span style="color: #483d8b;">'&lt;a href=&quot;(http://<span style="color: #000099; font-weight: bold;">\d</span>*<span style="color: #000099; font-weight: bold;">\.</span><span style="color: #000099; font-weight: bold;">\d</span>*<span style="color: #000099; font-weight: bold;">\.</span><span style="color: #000099; font-weight: bold;">\d</span>*<span style="color: #000099; font-weight: bold;">\.</span><span style="color: #000099; font-weight: bold;">\d</span>*/search<span style="color: #000099; font-weight: bold;">\?</span>q<span style="color: #000099; font-weight: bold;">\=</span>cache.*?)&quot;.*?&gt;Cached&lt;/a&gt;'</span><span style="color: black;">&#41;</span>
    regex_next = <span style="color: #dc143c;">re</span>.<span style="color: #008000;">compile</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'&lt;a href=&quot;([^&quot;]*?)&quot;&gt;&lt;span id=nn&gt;&lt;/span&gt;Next&lt;/a&gt;'</span><span style="color: black;">&#41;</span>
&nbsp;
    <span style="color: #808080; font-style: italic;">#this is the directory we will save files to</span>
    <span style="color: #ff7700;font-weight:bold;">try</span>:
        <span style="color: #dc143c;">os</span>.<span style="color: black;">mkdir</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'files'</span><span style="color: black;">&#41;</span>
    <span style="color: #ff7700;font-weight:bold;">except</span>:
        <span style="color: #ff7700;font-weight:bold;">pass</span>
    counter = <span style="color: #ff4500;">0</span>
    pagenum = <span style="color: #ff4500;">0</span>
    more = <span style="color: #008000;">True</span>
    <span style="color: #ff7700;font-weight:bold;">while</span><span style="color: black;">&#40;</span>more<span style="color: black;">&#41;</span>:
        pagenum += <span style="color: #ff4500;">1</span>
        <span style="color: #ff7700;font-weight:bold;">print</span> <span style="color: #483d8b;">&quot;PAGE &quot;</span>+<span style="color: #008000;">str</span><span style="color: black;">&#40;</span>pagenum<span style="color: black;">&#41;</span>+<span style="color: #483d8b;">&quot;: &quot;</span>+url
        req = <span style="color: #dc143c;">urllib2</span>.<span style="color: black;">Request</span><span style="color: black;">&#40;</span>url, <span style="color: #008000;">None</span>, headers<span style="color: black;">&#41;</span>
        page = <span style="color: #dc143c;">urllib2</span>.<span style="color: black;">urlopen</span><span style="color: black;">&#40;</span>req<span style="color: black;">&#41;</span>.<span style="color: black;">read</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        matches = regex_cache.<span style="color: black;">findall</span><span style="color: black;">&#40;</span>page<span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">for</span> match <span style="color: #ff7700;font-weight:bold;">in</span> matches:
            counter+=<span style="color: #ff4500;">1</span>
            tmp_req = <span style="color: #dc143c;">urllib2</span>.<span style="color: black;">Request</span><span style="color: black;">&#40;</span>match.<span style="color: black;">replace</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'&amp;amp;'</span>,<span style="color: #483d8b;">'&amp;'</span><span style="color: black;">&#41;</span>, <span style="color: #008000;">None</span>, headers<span style="color: black;">&#41;</span>
            tmp_page = <span style="color: #dc143c;">urllib2</span>.<span style="color: black;">urlopen</span><span style="color: black;">&#40;</span>tmp_req<span style="color: black;">&#41;</span>.<span style="color: black;">read</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
            <span style="color: #ff7700;font-weight:bold;">print</span> counter,<span style="color: #483d8b;">&quot;: &quot;</span>+match
            f = <span style="color: #008000;">open</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'files/'</span>+<span style="color: #008000;">str</span><span style="color: black;">&#40;</span>counter<span style="color: black;">&#41;</span>+<span style="color: #483d8b;">'.html'</span>,<span style="color: #483d8b;">'w'</span><span style="color: black;">&#41;</span>
            f.<span style="color: black;">write</span><span style="color: black;">&#40;</span>tmp_page<span style="color: black;">&#41;</span>
            f.<span style="color: black;">close</span><span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
        <span style="color: #808080; font-style: italic;">#now check if there is more pages</span>
        match = regex_next.<span style="color: black;">search</span><span style="color: black;">&#40;</span>page<span style="color: black;">&#41;</span>
        <span style="color: #ff7700;font-weight:bold;">if</span> match == <span style="color: #008000;">None</span>:
            more = <span style="color: #008000;">False</span>
        <span style="color: #ff7700;font-weight:bold;">else</span>:
            url = <span style="color: #483d8b;">&quot;http://www.google.com&quot;</span>+match.<span style="color: black;">group</span><span style="color: black;">&#40;</span><span style="color: #ff4500;">1</span><span style="color: black;">&#41;</span>.<span style="color: black;">replace</span><span style="color: black;">&#40;</span><span style="color: #483d8b;">'&amp;amp;'</span>,<span style="color: #483d8b;">'&amp;'</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #ff7700;font-weight:bold;">if</span> __name__==<span style="color: #483d8b;">&quot;__main__&quot;</span>:
    main<span style="color: black;">&#40;</span><span style="color: black;">&#41;</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># vim: ai ts=4 sts=4 et sw=4</span></pre></div></div>

<p>Before using the script you need to adjust the <code>search_term</code> variable near the beginning of the script. In this variable goes the search term for which all the available cache would be downloaded. E.g. to retrieve the cache of all the pages of http://www.example.org you should set <code>search_term</code> to <code>site:www.example.org</code></p>
]]></content:encoded>
			<wfw:commentRss>http://www.guyrutenberg.com/2008/10/02/retrieving-googles-cache-for-a-whole-website/feed/</wfw:commentRss>
		<slash:comments>63</slash:comments>
		</item>
	</channel>
</rss>

<!-- Dynamic Page Served (once) in 0.834 seconds -->

