Guy Rutenberg » Projects

Solving Sudoku using Python and Prolog

Guy — Thu, 01 Dec 2011 20:39:22 +0000

Two weeks ago, I add came up with an interesting algorithm for solving Hidato which basically involves decomposing the board the grid (can be square, hexagonal or any other shape), into classes of pieces and then arranging them (maybe I’ll write a detailed post on it in the future). So while pondering whether it would be interesting enough to go forward and actually implementing the algorithm compared to the work it would require, I started thinking what will be the simplest way to solve such puzzles, as opposed to efficient.

At first I’ve looked at general purpose constraint solvers, and decided to tackle Sudoku instead as it’s a bit simple to define in terms of constraints. I considered several libraries but in the end I’ve settled on plainly using Prolog. I chose Prolog because as a logic programming language, constraints are its bread and butter. I although kind of liked it as I haven’t done anything in Prolog for quite a few years.

Describing Sudoku in terms of constraints is extremely simple. You need to state that every cell is in a given range and that all rows, columns and sub-grid contain different integers. As mangling with lists in prolog isn’t fun, I’ve wrote a python program that outputs all the prolog statements with hardcoded references to the variables which build-up the board. It’s ugly but dead simple. The script gets the dimensions of the sub-grid.

import sys
 
VAR_TEMPLATE = "G%02d%02d"
 
def puzzle_declaration(x,y):
    lines = []
    for i in range(x*y):
        vars = map(lambda j: VAR_TEMPLATE % (i,j), range(x*y))
        lines.append('[' + ', '.join(vars)+ ']')
 
    print "puzzle("
    print '  ' + ',\n  '.join(lines)
    print "):-"
 
def constraints_range(x,y):
    vars = [VAR_TEMPLATE % (i,j) for i in range(x*y) for j in range(x*y)]
    print "\tVars = [" + ', '.join(vars) + "],"
    print "\tVars ins 1..%d," % (x*y)
 
def constraints_diff(x,y):
    # rows
    rows = []
    for i in range(x*y):
        row = [VAR_TEMPLATE % (i,j) for j in range(x*y)]
        rows.append("all_different([" + ', '.join(row) + "])")
 
    columns = []
    for j in range(x*y):
        column = [VAR_TEMPLATE % (i,j) for i in range(x*y)]
        columns.append("all_different([" + ', '.join(column) + "])")
 
    rects = []
    for i in range(x):
        for j in range(y):
            rect = [VAR_TEMPLATE % (k+y*i,l+x*j) for k in range(y) for l in range(x)]
            rects.append("all_different([" + ', '.join(rect) + "])")
 
    print '\t' + ',\n\t'.join(rows) + ","
    print '\t' + ',\n\t'.join(columns) + ","
    print '\t' + ',\n\t'.join(rects) + "."
 
def generate_puzzle(x,y):
    puzzle_declaration(x,y)
    constraints_range(x,y)
    constraints_diff(x,y)
 
 
if __name__=="__main__":
    x = int(sys.argv[1])
    y = int(sys.argv[2])
    generate_puzzle(x,y)

For example running the script with 3 3 as arguments generates a regular 9×9 Sudoku solver.

:- use_module(library(clpfd)).
 
puzzle(
  [G0000, G0001, G0002, G0003, G0004, G0005, G0006, G0007, G0008],
  [G0100, G0101, G0102, G0103, G0104, G0105, G0106, G0107, G0108],
  [G0200, G0201, G0202, G0203, G0204, G0205, G0206, G0207, G0208],
  [G0300, G0301, G0302, G0303, G0304, G0305, G0306, G0307, G0308],
  [G0400, G0401, G0402, G0403, G0404, G0405, G0406, G0407, G0408],
  [G0500, G0501, G0502, G0503, G0504, G0505, G0506, G0507, G0508],
  [G0600, G0601, G0602, G0603, G0604, G0605, G0606, G0607, G0608],
  [G0700, G0701, G0702, G0703, G0704, G0705, G0706, G0707, G0708],
  [G0800, G0801, G0802, G0803, G0804, G0805, G0806, G0807, G0808]
):-
	Vars = [G0000, G0001, G0002, G0003, G0004, G0005, G0006, G0007, G0008, G0100, G0101, G0102, G0103, G0104, G0105, G0106, G0107, G0108, G0200, G0201, G0202, G0203, G0204, G0205, G0206, G0207, G0208, G0300, G0301, G0302, G0303, G0304, G0305, G0306, G0307, G0308, G0400, G0401, G0402, G0403, G0404, G0405, G0406, G0407, G0408, G0500, G0501, G0502, G0503, G0504, G0505, G0506, G0507, G0508, G0600, G0601, G0602, G0603, G0604, G0605, G0606, G0607, G0608, G0700, G0701, G0702, G0703, G0704, G0705, G0706, G0707, G0708, G0800, G0801, G0802, G0803, G0804, G0805, G0806, G0807, G0808],
	Vars ins 1..9,
	all_different([G0000, G0001, G0002, G0003, G0004, G0005, G0006, G0007, G0008]),
	all_different([G0100, G0101, G0102, G0103, G0104, G0105, G0106, G0107, G0108]),
	all_different([G0200, G0201, G0202, G0203, G0204, G0205, G0206, G0207, G0208]),
	all_different([G0300, G0301, G0302, G0303, G0304, G0305, G0306, G0307, G0308]),
	all_different([G0400, G0401, G0402, G0403, G0404, G0405, G0406, G0407, G0408]),
	all_different([G0500, G0501, G0502, G0503, G0504, G0505, G0506, G0507, G0508]),
	all_different([G0600, G0601, G0602, G0603, G0604, G0605, G0606, G0607, G0608]),
	all_different([G0700, G0701, G0702, G0703, G0704, G0705, G0706, G0707, G0708]),
	all_different([G0800, G0801, G0802, G0803, G0804, G0805, G0806, G0807, G0808]),
	all_different([G0000, G0100, G0200, G0300, G0400, G0500, G0600, G0700, G0800]),
	all_different([G0001, G0101, G0201, G0301, G0401, G0501, G0601, G0701, G0801]),
	all_different([G0002, G0102, G0202, G0302, G0402, G0502, G0602, G0702, G0802]),
	all_different([G0003, G0103, G0203, G0303, G0403, G0503, G0603, G0703, G0803]),
	all_different([G0004, G0104, G0204, G0304, G0404, G0504, G0604, G0704, G0804]),
	all_different([G0005, G0105, G0205, G0305, G0405, G0505, G0605, G0705, G0805]),
	all_different([G0006, G0106, G0206, G0306, G0406, G0506, G0606, G0706, G0806]),
	all_different([G0007, G0107, G0207, G0307, G0407, G0507, G0607, G0707, G0807]),
	all_different([G0008, G0108, G0208, G0308, G0408, G0508, G0608, G0708, G0808]),
	all_different([G0000, G0001, G0002, G0100, G0101, G0102, G0200, G0201, G0202]),
	all_different([G0003, G0004, G0005, G0103, G0104, G0105, G0203, G0204, G0205]),
	all_different([G0006, G0007, G0008, G0106, G0107, G0108, G0206, G0207, G0208]),
	all_different([G0300, G0301, G0302, G0400, G0401, G0402, G0500, G0501, G0502]),
	all_different([G0303, G0304, G0305, G0403, G0404, G0405, G0503, G0504, G0505]),
	all_different([G0306, G0307, G0308, G0406, G0407, G0408, G0506, G0507, G0508]),
	all_different([G0600, G0601, G0602, G0700, G0701, G0702, G0800, G0801, G0802]),
	all_different([G0603, G0604, G0605, G0703, G0704, G0705, G0803, G0804, G0805]),
	all_different([G0606, G0607, G0608, G0706, G0707, G0708, G0806, G0807, G0808]).

Again, not so pretty, but it’s python-generated hence simple as hell. Now solving is easily by passing integers instead of the variables. You can see below a prolog solution that demos solving a Sudoku. Another nice thing, is that if there isn’t a unique solution, it will be output say which cells have definite value, and output the constraints that hold for the rest of the cells.

I’ve tested it with both regular size Sudokus and mini-size (6×6), and it seems to preform pretty fast.

?- puzzle(
  [G0000, G0001,     1,     3, G0004, G0005, G0006, G0007,     8],
  [G0100, G0101,     7,     5,     6, G0105, G0106,     4, G0108],
  [G0200,     3, G0202, G0203,     7,     8,     6, G0207, G0208],
  [G0300, G0301,     4,     2, G0304, G0305, G0306, G0307,     6],
  [G0400,     9, G0402, G0403, G0404, G0405, G0406,     5, G0408],
  [    1, G0501, G0502, G0503, G0504,     5,     9, G0507, G0508],
  [G0600, G0601, G0602,     1,     2, G0605, G0606,     6, G0608],
  [G0700,     1, G0702, G0703,     3,     7,     5, G0707, G0708],
  [G0800, G0801, G0802, G0803, G0804,     9,     3, G0807, G0808]
).
 
G0000 = 5,
G0001 = 6,
G0004 = 4,
G0005 = 2,
G0006 = 7,
G0007 = G0100, G0100 = 9,
G0101 = 8,
G0105 = 1,
G0106 = 2,
G0108 = 3,
G0200 = 4,
G0202 = 2,
G0203 = 9,
G0207 = 1,
G0208 = 5,
G0300 = 7,
G0301 = 5,
G0304 = 9,
G0305 = 3,
G0306 = 1,
G0307 = G0400, G0400 = 8,
G0402 = 3,
G0403 = 7,
G0404 = 1,
G0405 = 6,
G0406 = 4,
G0408 = G0501, G0501 = 2,
G0502 = 6,
G0503 = 4,
G0504 = 8,
G0507 = 3,
G0508 = 7,
G0600 = 3,
G0601 = 7,
G0602 = 5,
G0605 = 4,
G0606 = 8,
G0608 = 9,
G0700 = 6,
G0702 = 9,
G0703 = 8,
G0707 = 2,
G0708 = 4,
G0800 = 2,
G0801 = 4,
G0802 = 8,
G0803 = 6,
G0804 = 5,
G0807 = 7,
G0808 = 1.

passha – A hashapass variant

Guy — Sat, 17 Sep 2011 14:06:42 +0000

I like the idea of hashapass, but I’m unwilling to use an online tool, as I fear that someday it might be compromised. So I wrote down my own variant of hashapass. It uses slightly longer passwords, and sha256 as the hash function.

#! /usr/bin/python
 
"""
passha.py - Generate passwords from a master password and a parameter.
 
Based on hashapass (http://hashapass.com)
"""
import hmac
import hashlib
 
def main(passwd, param):
    hm = hmac.HMAC(passwd, param, hashlib.sha256)
    print hm.digest().encode("base64")[:10]
 
if __name__=="__main__":
    import getpass
    passwd = getpass.getpass()
    param = raw_input("Parameter: ")
    main(passwd, param)

CSS Compactor – Reduces CSS File Size

Guy — Sun, 16 Jan 2011 19:57:09 +0000

This is a script I wrote back 2006 that reduces the file size for CSS files by removing unnecessary whitespace and comments. It’s also capable of taking such compacted CSS file, and re-indent it to make it readable. For example it would take the following CSS:

/* sample css */
* {
    margin: 0px;
    padding: 0px;
}
 
/* define style for the logo */
#header .logo {
    float: left;
    /* another comment */
}

and turn it into:

*{ margin:0px; padding:0px;}#header .logo{ float:left;}

which is an equivalent but much shorter CSS code. It can also reindent it back to:

*{
	 margin: 0px;
	 padding: 0px;
}
#header .logo{
	 float: left;
}

As I said the script is very old, and was written around the time I learned how to use python, so it is far from perfect. But despite that, it does what it was written for and it’s useful, so I deemed it worth to release it. You can download it from here: csscompactor-0.4.tar.gz.

The code is released under the GPL, so feel free to modify it. If there will be enough interest in it, I’ll probably re-write so it would have prettier and more efficient code.

Building CookieJar out of Firefox’s cookies.sqlite

Guy — Sat, 27 Nov 2010 15:19:55 +0000

Firefox 3 started to store it’s cookies in a SQLite database instead of the old plain-text cookie.txt. While Python’s cookielib module could read the old cookie.txt file, it doesn’t handle the new format. The following python snippet takes a CookieJar object and the path to Firefox cookies.sqlite (or a copy of it) and fills the CookieJar with the cookies from cookies.sqlite.

import sqlite3
import cookielib
 
def get_cookies(cj, ff_cookies):
    con = sqlite3.connect(ff_cookies)
    cur = con.cursor()
    cur.execute("SELECT host, path, isSecure, expiry, name, value FROM moz_cookies")
    for item in cur.fetchall():
        c = cookielib.Cookie(0, item[4], item[5],
            None, False,
            item[0], item[0].startswith('.'), item[0].startswith('.'),
            item[1], False,
            item[2],
            item[3], item[3]=="",
            None, None, {})
        print c
        cj.set_cookie(c)

It works well for me, except that apperantly Firefox doesn’t save session cookies to the disk at all.

spass-2.0 – Secure Password Generator

Guy — Sat, 21 Aug 2010 11:44:14 +0000

This is a complete rewrite of my secure password generator. The new version uses my a true random number generator (and here).

The major change was using the new true random number generator in order to ensure strong passwords. Less significant changes include an easy way to specify password’s strips, and some calling convention changes.

Usage examples:

$ ./spass
E5pT35Fg
$ ./spass -l 14
R$tfOm4g_yRQ2J
$ ./spass -s 0-9a-f -l 32
8b5f14a1eeaabe58c2878ab5416a9ebb

Download the tarball spass-2.0.tar.bz2. The program depends on Boost‘s program_options (it was tested against version 1.37 and 1.42 and should work with other versions too).

Statistical Tests for My Audio Based Random Number Generator

Guy — Thu, 12 Aug 2010 21:37:25 +0000

In May I’ve written about a way to generate random number from audio noise. Basically it went like this:

Get audio sample from the microphone.
Push the least significant bit to a buffer.
Repeat steps 1-2 until the buffer is full (buffer size == block size for the hash function).
Apply the hash function on the buffer.
Get random bits from the digest.

In order to continue developing this random number generator (RNG), I’ve written a C++ class that simplifies working with it.

/*
 * Copyright (C) 2010  Guy Rutenberg
 * http://www.guyrutenberg.com
 * 
 * This program is free software: you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation, either version 3 of the License, or
 * (at your option) any later version.
 * 
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 * 
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see .
 */
 
#include 
#include "md5.h"
 
class Grandom {
	public:
		Grandom();
		virtual ~Grandom();
		/**
		 * Generate a random dword
		 */
		uint32_t operator()();
	private:
		void gather_entropy();
		void get_block();
 
		FILE* m_dsp_fd;
		uint32_t m_index;
		union {
			char digest[16];
			uint32_t v[4];
		} m_buffer;
 
		md5_ctx m_md5_ctx;
		uint32_t m_block[512/32];
};

The Grandom class uses MD5 as the hash function, 16-bit samples and 44.1KHz sampling rate. This configuration should work well on any modern sound card. The full implementation is available in grandom-1.0.tar.bz2, which is a small test app.

To have some evidence for the quality of the random number generation I’ve tested it against NIST’s statistical test suite. I’ve tested 320 streams of 1M bits each. The results showed that the random number generator passed all of the tests but one (one of the non-overlapping pattern test, although it was very close to pass). If you wan to see the results, take a look at the final analysis report.

Overall, Grandom seems to be a good way to generate small amounts of high quality random numbers.

Audio Based True Random Number Generator POC

Guy — Fri, 14 May 2010 12:18:14 +0000

Few days ago I came up with an idea to create a true random number generator based on noise gathered from a cheap microphone attached to my computer. Tests showed that when sampling the microphone, the least significant bit behaves pretty randomly. This lead me to think it might be good source for gathering entropy for a true random number generator.

The base design was to gather the noise from the microphone than apply a process that will make in more uniform and refine its randomness. After some design iterations I came up with a process based on applying a hash function to the noise. Each iteration involves filling block of the hash function from the least significant bits of the microphone output and applying the hash. Each iteration outputs the current hash digest. Assuming the hash function is uniform, this will output a uniformly distributed blocks of bits. Furthermore, because there the previous state of the hash function influences the next digest computation, the process accumulates entropy that can smooth out potentially less random blocks. Because for all commonly used hash function the block size is much larger than the digest size the output can tell much about the current state or any future or past state. This also holds true even if someone can find all pre-images of the hash function as the amount of possible states will be too big.

I’ve built a Python proof of concept (using md5 as a hash function) suitable for Linux.

import hashlib
import struct
 
 
class GRandom:
    def __init__(self):
        self.audio = open("/dev/dsp","rb")
        self.hash = hashlib.md5()
 
    def get_raw_block(self):
        buffer = self.audio.read(self.hash.block_size*8)
        bytes = struct.unpack("%iB"%len(buffer), buffer)
 
        longs = []
        for i in range(self.hash.block_size/4):
            temp = 0
            for b in bytes[i*32:(i+1)*32]:
                temp = (temp << 1) ^ (b & 1)
            longs.append(temp)
        return struct.pack("%iI"%len(longs), *longs)
 
    def get_block(self):
        self.hash.update(self.get_raw_block())
        return self.hash.digest()

The amount of generated bits per second is given by (sample rate)*(digest size)/(block size). So for 8KHz (default) sampling rate and md5 we’ll get a theoretical speed of 2000b/s. SHA type hashes have higher digest to block size ration thus may result in higher speeds. Another source of speed up may be to change the sample rate of the microphone. But setting it too high may have negative effects on the entropy. The code may get a considerable performance gain by porting it to c/c++, as it uses both bit manipulations and calculates hashes. Anyways, even the Python implementation’s speed allows us it be used for many cases where true randomness is required, such as generating passwords.

Update 2010-08-07 I’ve corrected the speed calculation.

Hash Puppy 0.2

Guy — Tue, 30 Mar 2010 21:15:06 +0000

This is an update for my simple easy-to-use checksum calculator. It supports md4, md5, and sha1 hash functions. I wrote the project as a way to experience and learn Qt.

Changes since the previous version (Hash Puppy 0.1) include ability to abort a checksum calculation and improved GUI responsiveness. Also there were other minor tweaks to make Hash Puppy easier to use.

The software is available under the GPL license.
Source code: hashpuppy-0.2.tar.gz
Binary for Windows: hashpuppy-0.2.zip (Compiled against Qt 4.6.2).

Improved FTP Backup for WordPress

Guy — Sun, 28 Feb 2010 06:38:44 +0000

This script backups both the database and files of a WordPress blog into a remote FTP server (while keeping a local copy). It’s an update of my WordPress Backup to FTP script. The main changes are auto-detecting database settings and better support for caching plugins (specifically WP-Cache). The new version makes it easier to backup multiple WordPress blogs to the same FTP server.

Usage is pretty simple after a short initial configuration. First, save the the script and make it executable:

chmod +x wp-backup

(assuming you saved it under the name wp-backup). After saving it edit the file with your favorite editor and set the 5 configuration variable to whatever is appropriate for you. BACKUP_DIR is the folder to save the local backups to. FTP_HOST, FTP_USER, FTP_PASS control the FTP host, username and password, respectively, for the remote backup server. FTP_BACKUP_DIR sets the folder on the FTP server to save the remote backup to.

Now that the initial configuration is done, all you need to do is execute the script and give the path to the blog as an argument. For example:

./wp-backup /home/someuser/myblog

And that it, the script will backup your files (excluding cache) and database to both a local and remote locations. This allows using the same script to backup multiple WordPress blogs, unlike the previous script which had to be modified for each blog.

And now the script itself:

#!/bin/bash
 
# Copyright 2008, 2010 Guy Rutenberg 
# WordPress FTP backup 2.0
#
# Easily backup wordpress instances via ftp.
#
# Change Log:
# ===========
# 2.0:
#  - Auto-detect database settings.
#  - Exclude cache data from backups.
 
BACKUP_DIR=
FTP_HOST=
FTP_USER=
FTP_PASS=
FTP_BACKUP_DIR=
 
# end of configuration - you probably don't need to touch anything bellow
 
PROG=`basename "$0"`
print_usage () {
    echo "USAGE: ${PROG} [options] BLOG_ROOT"
    echo "Backup a WordPress blog"
}
 
print_help ()  {
    print_usage
    cat << EOF
 
Options:
    -h, --help          show this help message and exit
 
EOF
}
 
TEMP=`getopt -o h --long help -n "$PROG" -- "$@"`
if (($?)); then
    print_usage
    exit 1
fi
 
eval set -- "$TEMP"
 
while true ; do
    case "$1" in
        -h|--help) print_help; exit ;;
        --) shift; break;;
    esac
done
 
if [ -z "$1" ]
then
 print_usage > /dev/stderr
 exit 1
fi
 
BLOG_DIR=$1
DB_NAME=`echo "\"${BLOG_DIR}/wp-config.php\"); echo DB_NAME;" | php`
DB_USER=`echo "\"${BLOG_DIR}/wp-config.php\"); echo DB_USER;" | php`
DB_PASS=`echo "\"${BLOG_DIR}/wp-config.php\"); echo DB_PASSWORD;" | php`
DB_HOST=`echo "\"${BLOG_DIR}/wp-config.php\"); echo DB_HOST;" | php`
 
BLOG_DIR=`dirname "$BLOG_DIR"`/`basename "$BLOG_DIR"`
BACKUP_DIR=`dirname "$BACKUP_DIR"`/`basename "$BACKUP_DIR"`
 
echo -n "dumping database... "
DUMP_NAME=${DB_NAME}-$(date +%Y%m%d).sql.bz2
mysqldump --user=${DB_USER} --password=${DB_PASS} --host=${DB_HOST} \
 --databases ${DB_NAME} \
 | bzip2 -c > ${BACKUP_DIR}/${DUMP_NAME}
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 1
fi
echo "done"
 
echo -n "Creating tarball... "
TAR_NAME=${BLOG_DIR##*/}-$(date +%Y%m%d).tar.bz2
tar -cjf ${BACKUP_DIR}/${BLOG_DIR##*/}-$(date +%Y%m%d).tar.bz2 --exclude cache ${BLOG_DIR}
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 2
fi
echo "done"
 
echo -n "Uploading SQL dump and tarball to FTP... "
lftp -u ${FTP_USER},${FTP_PASS} ${FTP_HOST} <
if [ "$?" -ne "0" ]; then
	echo "failed!"
	exit 3
fi
echo "done"

An Early Release of the New cssrtl.py-2.0

Guy — Sun, 20 Sep 2009 10:00:50 +0000

It has been three years since I’ve released the original version of cssrtl.py (and two since it’s re-release). The old version did a nice job, but experience gained during that time led me to write from scratch a new version. I’ve detailed more than a month ago, the basic principles and ideas that guided me to design a better tool to help adapting CSS files from left-to-right to right-to-left.

The guidelines weren’t just empty words, they were written while working on the Hebrew adaptation to the Fusion theme and in the same time writing a new proof-of-concept version of cssrtl.py. The original intent was to release a more mature version of that code when it will be completed. However, due to the apparent shortage of time in the present and foreseeable future, I can’t see myself complete the project any time soon. So following the “release early” mantra, I’ve decided to release the code as-is. As I said, the code is in working state, but not polished, so it may be of benefit but may contain bugs. If you find any bugs or have any suggestions, I would be glad to hear.

Download

You can download the new version from here: cssrtl.py-2.0.tar.bz2. The code is available under the GPLv2 or any later version.

Usage

The new version works by creating a seperate css “fix” file that fixes the directionality of the css styles.

./cssrtl.py < main.css > main-rtl.css

Afterward, just link the main-rtl.css after main.css in your HTML files.