Batch Renaming Using sed

I was reorganizing my music library and decided to change the naming convention I’ve used. This task is just asking to be automated. Since the filename change could be described using regular expression, I looked for a way to use sed for the renaming process.

The files I had, had the following pattern as filename ARTIST – SONG – TRACK – ALBUM

James Brown - I Got You (I Feel Good).ogg  - 01 - Classic James Brown

I wanted to rename it to ARTIST – ALBUM – TRACK – NAME

James Brown - Classic James Brown - 01 - I Got You (I Feel Good).ogg

Describing the change as a sed program is easy:

s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/

Now all that has to be done is to pass each filename to mv and pass it again after it went through the sed script. This could be done like this:

for i in *; do
  mv "$i" "`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`";

The important part is the

`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`

which pipes the filename to sed and returns it as an argument for mv.

To see what renaming will be done one can alter a bit the above command, and get

for i in *; do
  echo "$i" "->" "`echo $i | sed "s/\(.*\) - \(.*\) - \(.*\) - \(.*\).ogg/\1 - \4 - \3 - \2.ogg/"`";

While will effectively print a list of lines of the form oldname -> newname.

Of course this technique isn’t limited to the renaming I’ve done. By changing the pattern given to sed, one can do any kind of renaming that can be described as a regular expression replacement. Also one can change the globbing (the *) in the for loop to operate only on specific files, that match a given pattern, in the directory instead of all of them.

Deleting a Range of Tickets in Trac

Recently the Open Yahtzee website which runs Trac has fallen victim to several spam attacks. The spammers submit large number of tickets containing links to various sites. This post was written mainly to allow me to copy paste a command to delete a range of tickets at once, but I thought it may be useful to others as well.
Continue reading Deleting a Range of Tickets in Trac

Extract Public Key from X.509 Certificate as Hex

X.509 certificates are common way to exchange and distribute public key information. For example, most Open Social containers use the OAuth RSA-SHA1 signature method, and distribute their public keys in the X.509 format.

While working on an AppEngine application, I needed to verify requests from such containers. However, there is (currently) no pure python library able of parsing the certificates. This meant that I needed extract the public key out of the certificate manually, and store it in some parsed way inside the Python code.

Fortunately, parsing public keys form a X.509 certificate and representing them as a Hex number turned out simple and easy.
Continue reading Extract Public Key from X.509 Certificate as Hex

Expanding Macros into String Constants in C

Today I came across an annoying problem, how do I expand a C macro into a string?

One of C’s preprocessor operators is the # which surrounds the token that follows it in the replacement text with double quotes (“). So, at first the solution sounds pretty simple, just define

#define STR(tok) #tok

and things will work. However, there is one caveat: it will not work if passed another macro. For example,

#define BUF_LEN 100
#define STR(tok) #tok


will produce after going through the preprocessor


instead of "100", which is undesired. This behavior is due to the C standard noting that no macro expansions should happen to token preceded by #.

However, after reconsidering the source of the problem, I’ve found the following workaround: define another macro which will expand the argument and only then call the macro which does the quoting.

#define STR_EXPAND(tok) #tok
#define STR(tok) STR_EXPAND(tok)

#define BUF_LEN 100


will produce


as desired.

Explanation: The STR macro calls the STR_EXPAND macro with its argument. Unlike in the first example, this time the parameter is checked for macro expansions and evaluated by the preprocessor before being passed to STR_EXPAND which quotes it, thus giving the desired behavior.

Drawing Finite Automata and State Machines

I had to draw couple of Finite Automata and Turing Machines for some university assignments. Usually I would have done it using Inkscape (as it is my favorite tool for creating figures for my LaTeX documents), but doing it manually is pretty tedious work. Inkscape diagram tool is currently sub par, so everything have to be done by hand. It’s OK if you need to draw one State Machine once in a while, but not suitable for larger quantities. I’ve also tried using Dia, but it also required lots of manual tweaking and tuning.

To my surprise, Graphviz (and especially the dot utility) turned out to be the (almost) perfect tool for the job. It lets you describe the graph in a simple text-based way, and it handles the graph layout by himself. This is somewhat like LaTeX but for graphs (you concentrate on content not layout).

My Finite Automata needed no manual tweaking and resulted in a very nice graphs. For more complicated State Machines it’s sometimes necessary to do some manual tuning. The commands I found most useful to tweak the graph were:

  • Grouping nodes to be in the same level – { rank="same"; "q1"; "q2"; "q3"}. The other options for rank can affect how the group is positioned relative to the other nodes in the graph (source, above all, sink bellow all).
  • Adding weight to edges – q1 -> q2 [weight="10"]. This affects the cost of strecting the edge. The higher the weight the straighter the edges will be.
  • Adding invisible edges – q1 -> q3 [style="invis"]. This allowed me to control the order of the nodes in the same rank (height).

Last but not least Graphivz can generate the graphs in variety for formats including eps, pdf and svg (which allows post-processing with inkscape).

Alpha Channel Problems When Creating .ico Files Using ImageMagick

I’ve tried to use ImageMagick to create .ico files for Open Yahtzee, out of PNGs of various sizes. The command as it should have been:

convert openyahtzee16.png openyahtzee32.png openyahtzee64.png openyahtzee.ico

resulted in the alpha channel being reversed. I’ve used ImageMagick 6.4.0, and I didn’t remember this misbehavior happening in the previous versions.

While this annoyed, and was due to no apparent reason, it could be easily solved using the ImageMagick switches to reverse the alpha channel:

-channel Alpha -negate

So the command that produces a correct .ico file was:

convert openyahtzee16.png openyahtzee32.png openyahtzee64.png -channel Alpha -negate openyahtzee.ico

Generating URL List from Access Log (access_log)

I had to parse an access_log of a website, in order to generate a sitemap. More precisely, a list of all URLs in the site. After playing around I’ve found a solution using sed, grep, sort and uniq. The good thing that each of this tools is available by default on most Linux distributions.
Continue reading Generating URL List from Access Log (access_log)

rzip vs. bzip2 – A short comparison

I decided to benchmark rzip against bzip for my backup needs. The benchmark was performed on a 89M tar archive of a directory which I regularly backup using my Amazon S3 backup script. The directory contains mostly LaTeX, PDF and Open Office files, so this benchmark may reflect very different results than what you will get if you will test it on other kinds of files.
Continue reading rzip vs. bzip2 – A short comparison

usb 1-4: device descriptor read/64, error -71

When I try to connect my Sansa Clip MP3 player to the linux box I see the following error in dmesg:

usb 1-4: device descriptor read/64, error -71

and the device recognition fails. The player’s battery gets reloaded but I can’t mount it and transfer songs.
Continue reading usb 1-4: device descriptor read/64, error -71

Start Trac on Startup – Init.d Script for tracd

As part of a server move, I went on to reinstall Trac. I’ve tried to install it as FastCGI but I failed to configure the clean URLs properly. I got the clean URLs to work if the user access them, but Trac insisted on addeing trac.fcgi to the beginning of every link it generated. So I’ve decided to use the Trac standalone server, tracd.

The next problem I faced was how to start the Trac automatically upon startup. The solution was to use an init.d script for stating Trac. After some searching, I didn’t find an init.d script for tracd that were satisfactory (mostly poorly written). So I went on an wrote my own init.d script for tracd.
Continue reading Start Trac on Startup – Init.d Script for tracd