Starting Django upon Reboot using Cron

Three years ago I wrote about starting services as a user via cron instead of init.d (specifically Trac). However, this method has a serious downside: it has no support for dependencies. This really bothered me when I used cron to start a Django server, as it would attempt to load before MySQL was running. This made cron useless, as after every reboot I would receive an error email saying it couldn’t connect to the MySQL server, and I would have to log in and start the Django server manually. Yesterday I got sick of it and decided to hack something together that will work properly. So my crontab had the following line:

@reboot python ./manage.py runfcgi ...options...

which I changed to:

@reboot until /sbin/status mysql | grep start/running ; do echo "Mysql isn't running yet...>&2"; sleep 1; done; python ./manage.py runfcgi ...options...

Basically, it loops and checks whether the MySQL service has started. If so, it starts Django as it did before. On the other hand, if MySQL isn’t running, it just sleeps for a second and repeats the check.

A small issue is that if, for some reason, MySQL won’t start at all, it will loop forever. If this happens, it would mean that I’ll have to manually kill that cron job, but I would have to log in anyway to see what’s wrong with MySQL. So while this method can’t support dependencies like init.d does, it does provide a good-enough solution.

Update 2012-11-23: Fixed the crontab line (it would fail when mysql was in the start/post-start state).

Gmail backup: getmail vs. OfflineIMAP

I’m currently reviewing my backup plans and decided it’s a good occasion to finally start backing up my Gmail account. Firstly, I didn’t seriously consider desktop clients as the main backup tool, as they are hard to automate. The two main options are OfflineIMAP and getmail. Both are available from Ubuntu’s repositories, so installation is easy with both, and both have good tutorials: Matt Cutts’ getmail and EnigmaCurry’s OfflineIMAP.

OfflineIMAP claims to be faster, but I haven’t really checked it (and I’m not sure how important that is, given that it runs in the background). From what I saw, configuring them is mainly a task of cut-and-paste, but getmail requires you to list every label you want to back up, which I consider a major downside. As both are able to save the mail in maildir format, it should be easy to back it up using duplicity.

Conclusion: This was a short comparison, mainly to guide me in choosing the right backup for me. You may have different opinions (which, of course, I would gladly hear). I finally chose OfflineIMAP, mainly due to the labels issue.

Note on desktop clients: It seems that every decent one can be configured to work with a local maildir, so you can use them to read the backups. As I prefer Gmail’s interface, I will only use desktop clients in case I’m offline, so read-only access from a desktop client seems good enough for me.

The Annoying eBook vs. Paperback Pricing

I’m an avid Kindle user and have been for more than a year. However, once in a while, I come across something like this when I’m shopping for a new book:

As you can see, Amazon sells the Kindle edition for a higher price than a paperback. This book, of course, isn’t the only example of this ridiculous pricing method, and if one browses the Kindle store, he will surely find more.

This really upsets me, as there is no honest reason to price an electronic edition higher than a real dead-tree paper edition. In both cases, the author and the publisher get their royalties and share of the profits. But Kindle editions don’t have many related expenses, like storage, transportation (from the publisher to Amazon), and above all, printing costs.

I don’t know who is to blame for this absurd thing, Amazon or the publisher (or even both). But the few things I know are that this is bad for everyone: the customer, because he pays more, and Amazon/the publisher, as in the long run, this encourages piracy, because the customer feels he’s being unfairly treated, and thus he will be more willing to play an unfair game as well.

Creating a Deb for an Updated Version

Say you’ve an existing package like gitg and you want to use a new version of gitg or even apply your own patches. You could directly make install, but you will probably regret it as soon as you’ll want to upgrade or uninstall, and you’ll want to create a better package than the one created by checkinstall. Apparently, creating a deb package for a new version of an already packaged deb isn’t complicated.

Getting Started

Start by pulling the sources for the already available package. I’ll be using gitg as an example throughout this tutorial.

$ apt-get source gitg

This will create a folder according to the version of the package, something like gitg-0.2.4. Extract the new version beside it and cd into its directory. The next step is to copy the debian/ directory from the old source package to the code you’ve just extracted.

$ cp -R ../gitg-0.2.4/debian/ .

Dependencies

There are some dependencies you’ll need:

$ sudo apt-get install devscripts
$ sudo apt-get install dpkg-dev

You’ll probably want to do:

$ sudo apt-get build-dep gitg

in order to make sure you’ve all the relevant build-time dependencies.

Update debian/ Files

The next step is to update the files under the debian/ sub-directory.

$ DEBEMAIL="Guy Rutenberg <myemail@domain.com>" debchange --nmu

This will update the debian/changelog and set the new version. --nmu will create a new “non-maintainer upload” version, meaning if the current version was 0.2.4-0ubuntu1, it will change it to 0.2.4-0ubuntu1.1. This will make sure that there won’t be any collision between your package and an official one. If you update to a new upstream version, it might be more suitable to use something like this:

$ debchange --newversion 0.2.5+20111211.git.20391c4

If necessary, update the Build-Depends and Depends sections of debian/control.

Building the Package

If you’re building a package directly from version control and not as part of an official release, you may need to run

$ ./autogen

at this point.

Now to the actual building:

$ debuild -us -uc -i -I -b

-us -uc tells the script not to sign the .dsc and .changes files accordingly. -i and -I make the script ignore common version control files. -b tells debuild to only create binary packages. You can also pass -j followed by the number of simultaneous jobs you wish to allow (e.g. -j3, like in make), which can significantly speed things up.

Installing the Package

The package will reside in the parent directory, for example:

../gitg_0.2.5+20111211.git.20391c4_amd64.deb

At this point you’re basically done. If you want to install the package, you can use

sudo debi

while you’re still inside the build directory.

Creating Source Packages

If you want to go the extra mile and create source packages, it will make things easier for others to build their own packages based on yours.

You’ll need to create an “orig” tarball

../gitg_0.2.5+20111211.git.20391c4.orig.tar.bz2

(note the underscore between the package name and the version). The “orig” tarball should contain the original source code without the Debian-specific patches.

Now you can run the debuild command like before but without the -b flag.
This will create the following files:

../gitg_0.2.5+20111211.git.20391c4.debian.tar.gz
../gitg_0.2.5+20111211.git.20391c4.dsc

References

  • UpdatingADeb
  • Man pages for debuild, dpkg-genchanges, dpkg-buildpackage.

Using gitg without installing

I’m working on adding spell-checking support to gitg. If you intend to use gitg without installing it, a little hack is necessary. You’ll need to symlink the gitg directory (the one with the source files) as ui.

ln -s gitg ui
./configure /pathto/below/gitg

The reason is that gitg will look for Glade UI files under $(datadir)/gitg/ui, and in gitg’s source the UI files are in the gitg directory and not in ui.

You can see above a screenshot of gitg with spell-checking enabled. Hopefully I’ll be done with the changes soon, and they will be merged upstream quickly.

Update: There are a couple more things to do in order to get GSettings’ schemas right.

mkdir glib-2.0
ln -s ../data glib-2.0/schemas
glib-compile-schemas data/
XDG_DATA_DIRS=".:/usr/share/" ./gitg/gitg

For the schemas thing, see glib-compile-schemas‘ man page.

Update 2011-12-17: Jesse (gitg’s maintainer) hasn’t responded to my email regarding the new feature, so I’ve opened a bug (#666406) for it. If you’re willing to try the changes yourself, you can pull them from git://github.com/guyru/gitg.git spellchecker.

GNOME_COMPILE_WARNINGS(maximum) – Syntax Error in configure

I’m still encountering migration issues from Gentoo to Ubuntu. Apparently, Gentoo is much more user-friendly than Ubuntu when it comes to compiling packages. In Gentoo, you’ve got almost all the major dependencies you need. In Ubuntu, on the other hand, you need to hunt them down. It’s much easier with the main ones, as they are listed. But there are some small ones that are harder to track. I came across the following error while trying to compile gitg, a GUI for Git, today:

./configure: line 14447: syntax error near unexpected token `maximum'
./configure: line 14447: `GNOME_COMPILE_WARNINGS(maximum)'

After a not-so-short investigation, I found out I was missing gnome-common.

sudo apt-get install gnome-common

Why can’t there be one distribution that is user-friendly like Ubuntu and, at the same time, developer-friendly like Gentoo?

Solving Sudoku using Python and Prolog

Two weeks ago, I came up with an interesting algorithm for solving Hidato, which basically involves decomposing the board grid (which can be square, hexagonal, or any other shape) into classes of pieces and then arranging them (maybe I’ll write a detailed post on it in the future). So while pondering whether it would be interesting enough to go forward and actually implement the algorithm, compared to the work it would require, I started thinking about what would be the simplest way to solve such puzzles, as opposed to the most efficient way.

At first I looked at general-purpose constraint solvers and decided to tackle Sudoku instead, as it’s a bit simpler to define in terms of constraints. I considered several libraries, but in the end I settled on simply using Prolog. I chose Prolog because, as a logic programming language, constraints are its bread and butter. I also kind of liked it, as I haven’t done anything in Prolog for quite a few years.

Describing Sudoku in terms of constraints is extremely simple. You need to state that every cell is in a given range and that all rows, columns, and sub-grids contain different integers. Since mangling with lists in Prolog isn’t fun, I wrote a Python program that outputs all the Prolog statements with hardcoded references to the variables that build up the board. It’s ugly but dead simple. The script gets the dimensions of the sub-grid.
Continue reading Solving Sudoku using Python and Prolog

GCC Usage Quirks

This is more of a note to myself, since it’s an error I keep bumping into. When compiling manually using gcc (or g++), you should pass all the libraries you’re compiling against at the end of the argument list. For example:

g++ -Wall -lboost_thread my_file.cpp

will result in an error like:

undefined reference to `boost::thread::join()'
undefined reference to `boost::thread::~thread()'

while

g++ -Wall vortex5_brute.cpp -lboost_thread

works fine.

Number Exercises Separately in LyX

Say you’ve got a document with a bunch of exercises and a few lemmas. You may want the exercises numbered separately from the lemmas and theorems, unlike LyX’s default behavior. This can be achieved by redefining xca, the environment LyX uses for exercises. Add the following to your LaTeX preamble:

letxca@undefined
theoremstyle{plain}
newtheorem{xca}{protectexercisename}

LyX will still display the incorrect numbering, but the output will be correct nonetheless. The first line undefines LyX’s definition of xca, then we set the style to match the old one, and we redefine xca, this time without a reference to the theorem counter.

Expectation Symbol in LaTeX

After looking for a built-in expectation symbol in LaTeX, and coming up with none, I’ve defined one. Just add:

% Expectation symbol
DeclareMathOperator*{E}{mathbb{E}}

to your LaTeX preamble, and you’re done. You’ll also need to add usepackage{amsmath}, or in LyX, tick “Use AMS math package” under Document->Settings->Math Options.

Using the starred version of DeclareMathOperator makes sure subscripts go beneath the symbol in display mode.