The major change was using the new true random number generator in order to ensure strong passwords. Less significant changes include an easy way to specify password’s strips, and some calling convention changes.

Usage examples:

$ ./spass
E5pT35Fg
$ ./spass -l 14
R$tfOm4g_yRQ2J
$ ./spass -s 0-9a-f -l 32
8b5f14a1eeaabe58c2878ab5416a9ebb

Download the tarball spass-2.0.tar.bz2. The program depends on Boost‘s program_options (it was tested against version 1.37 and 1.42 and should work with other versions too).

1. Get audio sample from the microphone.
2. Push the least significant bit to a buffer.
3. Repeat steps 1-2 until the buffer is full (buffer size == block size for the hash function).
4. Apply the hash function on the buffer.
5. Get random bits from the digest.

In order to continue developing this random number generator (RNG), I’ve written a C++ class that simplifies working with it.

/*
 * Copyright (C) 2010  Guy Rutenberg
 * http://www.guyrutenberg.com
 * 
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
 
#include <cstdio>
#include "md5.h"
 
class Grandom {
	public:
		Grandom();
		virtual ~Grandom();
		/**
		 * Generate a random dword
		 */
		uint32_t operator()();
	private:
		void gather_entropy();
		void get_block();
 
		FILE* m_dsp_fd;
		uint32_t m_index;
		union {
			char digest[16];
			uint32_t v[4];
		} m_buffer;
 
		md5_ctx m_md5_ctx;
		uint32_t m_block[512/32];
};

The Grandom class uses MD5 as the hash function, 16-bit samples and 44.1KHz sampling rate. This configuration should work well on any modern sound card. The full implementation is available in grandom-1.0.tar.bz2, which is a small test app.

To have some evidence for the quality of the random number generation I’ve tested it against NIST’s statistical test suite. I’ve tested 320 streams of 1M bits each. The results showed that the random number generator passed all of the tests but one (one of the non-overlapping pattern test, although it was very close to pass). If you wan to see the results, take a look at the final analysis report.

Overall, Grandom seems to be a good way to generate small amounts of high quality random numbers.

The solution is to directly delete them from Trac’s SQLite backend. There are several type of comments: Plain ones and comments that describe a change to the ticket’s properties. I’ll deal with only the plain comments, but the methods I describe should also be applicable to the other kinds.

First things you should note down the ticket number and the comment number. The comment number appears in the URL of the permalink to it. It is the number that is pointed by an arrow in the screenshot above, it will be displayed when you hover over the circled link. Now in the shell, cd to the db directory of you trac installation and do:

sqlite3 trac.db "delete from ticket_change where ticket=<TICKET NUM> and field='comment' and oldvalue=<COMMENT NUM>"

For example for the ticket in the screenshot:

sqlite3 trac.db "delete from ticket_change where ticket=12 and field='comment' and oldvalue=3"

You can delete different kinds of tickets by using other values in the field column, such as keywords and summary. Another useful column is the author column.

I’ve built a Python proof of concept (using md5 as a hash function) suitable for Linux.

import hashlib
import struct
 
 
class GRandom:
    def __init__(self):
        self.audio = open("/dev/dsp","rb")
        self.hash = hashlib.md5()
 
    def get_raw_block(self):
        buffer = self.audio.read(self.hash.block_size*8)
        bytes = struct.unpack("%iB"%len(buffer), buffer)
 
        longs = []
        for i in range(self.hash.block_size/4):
            temp = 0
            for b in bytes[i*32:(i+1)*32]:
                temp = (temp << 1) ^ (b & 1)
            longs.append(temp)
        return struct.pack("%iI"%len(longs), *longs)
 
    def get_block(self):
        self.hash.update(self.get_raw_block())
        return self.hash.digest()

The amount of generated bits per second is given by (sample rate)*(digest size)/(block size). So for 8KHz (default) sampling rate and md5 we’ll get a theoretical speed of 2000b/s. SHA type hashes have higher digest to block size ration thus may result in higher speeds. Another source of speed up may be to change the sample rate of the microphone. But setting it too high may have negative effects on the entropy. The code may get a considerable performance gain by porting it to c/c++, as it uses both bit manipulations and calculates hashes. Anyways, even the Python implementation’s speed allows us it be used for many cases where true randomness is required, such as generating passwords.

Update 2010-08-07 I’ve corrected the speed calculation.

TypeError: character mapping must return integer, None or unicode

This is pretty unhelpful error message also occurs if you try any method that indirectly use altchars. For example:

base64.urlsafe_b64decode(unicode('aass'))
base64.b64decode(unicode('aass'),'-_')

both fail while the following works:

base64.urlsafe_b64decode('aass')
base64.b64decode(unicode('aass'))

While it’s not complicated to fix it (just convert any unicode string to ascii string), it’s still annoying.

The following Python functions allow you to build and read these 6 chars URL-safe timestamps:

import base64
import struct
import time
 
def build_timestamp():
    """
    Return a 6 chars url-safe timestamp
    """
    return base64.urlsafe_b64encode(struct.pack("!L",int(time.time())))[:-2]
 
def read_timestamp(t):
    """
    Convert a 6 chars url-safe timestamp back to time
    """
    return struct.unpack("!L",base64.urlsafe_b64decode(t+"=="))[0]

These functions work by translating the timestamp into a 4-byte binary form and then encoding it using a URL-safe version of Base64. And finally we strip the padding, which is neither URL-safe nor necessary (as we know the size of the encoded data).

The result looks something like this:

In [72]: build_timestamp()
Out[72]: 'S9sNOQ'

We got a timestamp in using only 6 URL-safe chars.

chmod +x wp-backup

(assuming you saved it under the name wp-backup). After saving it edit the file with your favorite editor and set the 5 configuration variable to whatever is appropriate for you. BACKUP_DIR is the folder to save the local backups to. FTP_HOST, FTP_USER, FTP_PASS control the FTP host, username and password, respectively, for the remote backup server. FTP_BACKUP_DIR sets the folder on the FTP server to save the remote backup to.

Now that the initial configuration is done, all you need to do is execute the script and give the path to the blog as an argument. For example:

./wp-backup /home/someuser/myblog

And that it, the script will backup your files (excluding cache) and database to both a local and remote locations. This allows using the same script to backup multiple WordPress blogs, unlike the previous script which had to be modified for each blog.

And now the script itself:

#!/bin/bash
 
# Copyright 2008, 2010 Guy Rutenberg <http://www.guyrutenberg.com/contact-me>
# WordPress FTP backup 2.0
#
# Easily backup wordpress instances via ftp.
#
# Change Log:
# ===========
# 2.0:
#  - Auto-detect database settings.
#  - Exclude cache data from backups.
 
BACKUP_DIR=
FTP_HOST=
FTP_USER=
FTP_PASS=
FTP_BACKUP_DIR=
 
# end of configuration - you probably don't need to touch anything bellow
 
PROG=`basename "$0"`
print_usage () {
    echo "USAGE: ${PROG} [options] BLOG_ROOT"
    echo "Backup a WordPress blog"
}
 
print_help ()  {
    print_usage
    cat << EOF
 
Options:
    -h, --help          show this help message and exit
 
EOF
}
 
TEMP=`getopt -o h --long help -n "$PROG" -- "$@"`
if (($?)); then
    print_usage
    exit 1
fi
 
eval set -- "$TEMP"
 
while true ; do
    case "$1" in
        -h|--help) print_help; exit ;;
        --) shift; break;;
    esac
done
 
if [ -z "$1" ]
then
 print_usage > /dev/stderr
 exit 1
fi
 
BLOG_DIR=$1
DB_NAME=`echo "<?php require_once(\"${BLOG_DIR}/wp-config.php\"); echo DB_NAME;" | php`
DB_USER=`echo "<?php require_once(\"${BLOG_DIR}/wp-config.php\"); echo DB_USER;" | php`
DB_PASS=`echo "<?php require_once(\"${BLOG_DIR}/wp-config.php\"); echo DB_PASSWORD;" | php`
DB_HOST=`echo "<?php require_once(\"${BLOG_DIR}/wp-config.php\"); echo DB_HOST;" | php`
 
BLOG_DIR=`dirname "$BLOG_DIR"`/`basename "$BLOG_DIR"`
BACKUP_DIR=`dirname "$BACKUP_DIR"`/`basename "$BACKUP_DIR"`
 
echo -n "dumping database... "
DUMP_NAME=${DB_NAME}-$(date +%Y%m%d).sql.bz2
mysqldump --user=${DB_USER} --password=${DB_PASS} --host=${DB_HOST} \
 --databases ${DB_NAME} \
 | bzip2 -c > ${BACKUP_DIR}/${DUMP_NAME}
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 1
fi
echo "done"
 
echo -n "Creating tarball... "
TAR_NAME=${BLOG_DIR##*/}-$(date +%Y%m%d).tar.bz2
tar -cjf ${BACKUP_DIR}/${BLOG_DIR##*/}-$(date +%Y%m%d).tar.bz2 --exclude cache ${BLOG_DIR}
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 2
fi
echo "done"
 
echo -n "Uploading SQL dump and tarball to FTP... "
lftp -u ${FTP_USER},${FTP_PASS} ${FTP_HOST} <<EOF
cd "${FTP_BACKUP_DIR}"
put "${BACKUP_DIR}/${DUMP_NAME}"
put "${BACKUP_DIR}/${TAR_NAME}"
 
EOF
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 3
fi
echo "done"

<form>
...
    <input type="hidden" name="to" value="support@example.com" />
...
</form>

as it is likely that if the value of the “to” field changes the message will be sent to the modified address. The problem with this kind of functionality is that it allows a malicious user to send emails from your mail server. More specifically, it can allow spammers to user your benign server t send their spam (and as a side effect you might be flagged as a spammer yourself).

As this case is pretty obvious one doesn’t see many real-life uses of it anymore (but careless programmers used it more often n the past until they learned better). However one can achieve similar goals (spam-wise) by utilizing a common feature in contact forms: the “CC yourself” checkbox.


“CC yourself” is a convention used by some people when mailing, to verify that the email was indeed sent. It had found a place in many contact forms, as people wanted a way to make sure the form indeed works. But contact forms (as well as some mail servers) don’t verify that the email provided as the “from” is indeed owned by whoever fills the form. Combine that with the fact that many contact forms don’t employ CAPTCHAs (to make the form simpler to use), and you’ll get a situation much like the one discussed above.

In the first case there was a usability advantage to the programmer (who could easily re-use the form’s backend for other forms), which can be easily sacrificed for enhanced security. This time it’s worse as this is a usability feature for the user, which many people believe to be very convinient in contact forms.

I think there are several solutions possible:

1. Adding CAPTCHA to the form. This will make life for the spammers harder, but it also hurt the users by raising the bar for filling out the form. Also, nowadays, it getting harder and harder to find a strong yet easy for humans CAPTCHA.
2. Removing the “CC yourself” feature. this hurts the usability of the contact form.
3. Seperating verified users and unverified. Keeping the feature for registered users, but at the same time, removing it or adding CAPTCHA for unverified users. This seems like a good trade-off, but it requires more work and registration is not applicable for all websites.

In my opinion none of the solutions is perfect. It seems that once again spammers hurt the user experience for everybody else in order to fill our inboxes with unsolicited email.