Guy Rutenberg » Tutorials

Creating a Deb for an Updated Version

Guy — Sat, 17 Dec 2011 16:16:04 +0000

Say you’ve an existing package like gitg and you want to use the new version of gitg or even apply your own patches. You could directly make install but you will probably regret it as soon as you’ll want to upgrade/uninstall, and you want to create a better package than the one created by checkinstall. Apperantly, creating a deb package for a new version of already packaged deb isn’t complicated.

Getting Started

Start by pulling the sources for the already available package. I’ll by using gitg as an example throughout this tutorial.

apt-get source gitg

This will create a folder according to the version of the package, something like gitg-0.2.4. Extract the new version besides it and cd into its directory. The next step is to copy the debian/ directory from the old source package the code you’ve just extracted.

cp -R ../gitg-0.2.4/debian/ .

Update `debian/` Files

The next step is to update the files under the debian/ sub-directory.

DEBEMAIL="Guy Rutenberg " debchange --nmu

This will update the debian/changelog and set the new version. --nmu will create a new “non maintainer upload” version, meaning if the current version was 0.2.4-0ubuntu1, it will change it to 0.2.4-0ubuntu1.1. This will make sure that there won’t be any collision between your package and an official one. If you update to a new upstream version. It might be more suitable to use something like this:

debchange --newversion 0.2.5+20111211.git.20391c4

If necessary, update the Build-Depends and Depends sections of debian/control.

Building the Package

If your building a package directly from version control and not part of an official release, you may need to run

./autogen

at this point.

Now to the actual building:

debuild -us -uc -i -I -B

-us -uc tells the script not to sign the .dsc and .changes files accordingly. -i and -I makes the script ignore common version control files. -B tells debuild to only create binary packages. You can also pass -j followed by the number of simultaneous jobs you wish to allow (e.g. -j3, like in make) which an significantly speed things up.

Installing the Package

The package will reside in the parent directory, for example:

../gitg_0.2.5+20111211.git.20391c4_amd64.deb

At this point you’re basically done. If you want to install the package you can use

sudo debi

while you’re still inside the build directory.

References

UpdatingADeb
Man pages for debuild, dpgk-genchanges, dpgk-buildpackage.

WordPress Administration over SSL on Lighttpd

Guy — Fri, 03 Dec 2010 09:46:07 +0000

In this tutorial we’ll walk through the steps of enabling SSL (https) for the WordPress’ admin panel when using Lighttpd as a webserver. The tutorial consists of two stages, the first is enabling SSL at the Lighttpd level and the second is in the WordPress level.

Lighttpd Setup

The first thing to do is to check that your lighttpd does support SSL. This should be the case for most distros. Look for the (ssl) after the version string.

$ lighttpd -v
lighttpd/1.4.26 (ssl) - a light and fast webserver
Build-Date: Jul 17 2010 15:02:18

We’ll generate a self-signed SSL certificate using openssl.

/etc/lighttpd$ sudo openssl req -new -x509 -newkey rsa:2048 -keyout server.pem -out server.pem -days 365 -nodes

It will ask you to provide some details. You should specify your domain name in the Common Name section. If you would like to use the domain for multiple subdomains (like guyrutenberg.com and www.guyrutenberg.com), you can use wildcards, e.g. *.guyrutenberg.com.

Now you should change the permissions of the certificate so it’s read-only and accessible by the root user:

/etc/lighttpd$ sudo chmod 0400 server.pem

Now you’re ready to enable the SSL in the Lighttpd’s configuration. Edit /etc/lighttpd/lighttpd.conf and insert the following section:

$SERVER["socket"] == ":443" {
    ssl.engine                  = "enable" 
    ssl.pemfile                 = "/etc/lighttpd/server.pem" 
}

Now make sure you didn’t do any dumb typing/copying mistake and restart the lighttpd service.

$ lighttpd -tf /etc/lighttpd/lighttpd.conf
$ sudo /etc/init.d/lighttpd restart

Now everything should be set correctly. However, if you experience the following error (happened on my Ubuntu box)

 * Starting web server lighttpd
2010-12-03 08:30:05: (network.c.336) SSL: error:00000000:lib(0):func(0):reason(0) 
   ...fail!

you should add the following lines to the socket section you’ve added to the lighttpd.conf:

    ssl.use-sslv2 = "enable"
    ssl.cipher-list = "TLSv1+HIGH RC4+MEDIUM !SSLv2 !3DES !aNULL @STRENGTH"

The workaround was taken from here.

Now last, but not least, don’t forget to let HTTPS (port 443) thourgh your firewall.

$ sudo ufw allow https

WordPress Setup

This part is pretty short. Basically you have two options:

Enabling SSL for the whole admin session.
Enabling SSL only for login.

The former is more secure, while the latter is easier on server performance-wise. I would recommend the former, unless you find it very slow. Enabling SSL for the whole session can be done by adding the following line

define('FORCE_SSL_ADMIN', true);

to your wp-config.php. Enabling SSL only for login is done by adding

  define('FORCE_SSL_LOGIN', true);

instead.

If you’ve followed so far, you should be done and able to access your WordPress admin panel in a secure manner.

Further resources:
http://redmine.lighttpd.net/wiki/1/Docs:SSL
http://codex.wordpress.org/Administration_Over_SSL

Capturing Video and Converting to H.264 using ffmpeg

Guy — Fri, 10 Sep 2010 16:21:15 +0000

8-millimeter video tapes seem to slowly fade to oblivion. In order to save old family videos recorded in this format, I’ve decided to digitize them.

After a quick try with vlc, I’ve understood that it wasn’t the right tool for the task. It crashed with a cryptic error message every time I’ve tried to encode H.264 video, and it seemed that it best suited for real time encoding. Doing real time encoding, is sub-optimal as I can’t reach high quality encoding is a reasonable bit rate.

So I looked for another tool and recalled ffmpeg. While ffmpeg provided everything I looked: high quality video encoding using H.264 and stability, it wasn’t an easy start. ffmpeg’s defaults are notoriously ill-chosen. After hours of going through man pages, I’ve managed to capture and convert video tapes into high quality (encoded) digital video.

Basically the process involved capturing the raw video into a temporary file and then preform a two-pass encoding using H.264.

Capturing the raw video

ffmpeg -i /dev/video0 -t 1:30:00 -vcodec copy /tmp/video.mpg

The -vcodec copy tells ffmpeg to copy the raw codec data as is. It allows you to make an identical copy of the video from the capturing device /dev/video0. The -t flag allows you to restrict the video capturing duration.

Performing the first encoding pass

ffmpeg -i /tmp/video.mpg -vcodec libx264 -vpre max -threads 0 \
-b 2100k -pass 1 -f mp4 -y /dev/null

The -vcodec libx264 tells ffmpeg to encode the video using H.264 encoding. The -vpre flag allows you to set a predefind presets that control (among other things) the quality of the encoded value. I used max, other presets include hq and normal. -threads 0 tells ffmpeg to spawn as many threads it deems necessary. -b 2100k tells it to target a bit rate of 2100kbps for video encoding. -pass 1 allows you to perform the first pass out two in a 2-pass encoding. In the first pass ffmpeg will collect statistics and write them to a special log. When doing the second pass ffmpeg can use this special log to better distribute the bit rate so it will achieve a constant quality at the specified average bit rate. The output of the first pass isn’t important so it’s directed to /dev/null. The -y switch tells ffmpeg to ignore that this file already exists. As ffmpeg can’t tell the format of the file by it’s name (/dev/null) we specify it manually using -f mp4.

Performing the final encoding pass

ffmpeg -i /tmp/video.mpg -vcodec libx264 -vpre max -threads 0 \
-b 2100k -pass 2 ~/Videos/video-200108-200201-r1.mp4

This one is similiar to the last command except that we finally save the video. The -pass 2 tells ffmpeg to perform the second and final pass. It will use the statistics log created by the previous command in order to optimize the encoding. The -f flag isn’t necessary as the format is indicated by the output file’s name.

The encoding time is greatly effected by the preset that you chose. The max preset is very slow, which wasn’t a problem for me in this case, but is something worth remembering. See Julian Simon’s x264 presets comparison for more information regarding the different presets available.

phpMyAdmin + Lighttpd in Gentoo

Guy — Tue, 20 Jan 2009 16:14:57 +0000

Usually installing software in Gentoo is a piece of cake. Just emerge what you want and (with the right USE flags) and everything will be ready for you. However, as today I’ve found out today, installing phpMyAdmin with Lighttpd isn’t trivial as it should be.

In this post I’ll try to walk you through the necessary steps to install phpMyAdmin with Lighttpd in Gentoo.

Installing Lighttpd and PHP

Before installing Lighttpd and PHP you’ll need to enable some USE flags. For Lighttpd you’ll need to at least enable the php and fastcgi USE flags. For PHP enable the cgi, crypt, ctype, pcre, session, unicode. The first one is required for working with Lighttpd, while the others are required for phpMyAdmin.

See the handbook for information about enabling USE flags.

Now install the packages

emerge -av lighttpd php

After installation is complete, start the Lighttpd process

/etc/init.d/lighttpd start

and you should have a working Lighttpd server with PHP support.

Installing phpMyAdmin

phpMyAdmin depends on a package called webapp-config. While usually we let Portage take care of dependencies, this time we’ll have to install webapp-config manually before we install phpMyAdmin, so we can change its configuration for suit Lighttpd.

emerge -av --oneshot webapp-config

After installation is done edit using your favourite editor the /etc/vhosts/webapp-config and change the line:

vhost_server="apache"

vhost_server="lighttpd"

Now we are ready to install phpMyAdmin.

emerge -av phpmyadmin

After installation is complete, you’ll need to create a phpMyAdmin configuration file

cp /var/www/localhost/htdocs/phpmyadmin/config.sample.inc.php /var/www/localhost/htdocs/phpmyadmin/config.inc.php

Now open the newly created configuration file and edit the line with the blowfish_secret (at the top of the file) as explained in it. An easy way to generate the password would be to use spass.

Now if you use the default Lighttpd configurations you’re done. You can access the phpMyAdmin via http://localhost/phpmyadmin. If however you’ve changed the document root, there is one more step for you. Enable the mod_alias module in /etc/lighttpd/lighttpd.conf and append to the file

alias.url += ("/phpmyadmin" => "/var/www/localhost/htdocs/phpmyadmin")

and don’t forget to restart the Lighttpd server after making the changes.

You should have by now everything installed and configured to work. If you have any further questions don’t hesitate to comment.

Understanding load average – A Practitioner Guide

Guy — Fri, 25 Jul 2008 08:50:34 +0000

The term “load average” is used in many Linux/UNIX utilities. Everybody knows that the numbers the term “load average” refers to, usually three numbers, somehow represent the load on the system’s CPU. In this post I’ll try making this three numbers clearer and understandable.

The easiest way to see the “load average” of your system is by uptime.It also appears in top and can be graphed in the console by tload . In all three cases the load average refers to a group of three numbers. For example, in the following output of uptime

10:41:47 up 5 days, 48 min,  1 user,  load average: 0.82, 0.71, 0.66

the last three numbers are the “load average”. Each number represent the systems load as a moving average over 1, 5 and 15 minutes respectively. Now, the important thing is to understand what is being averaged, the load metric.

The metric that represent the load at a given point in time is how many process are queued for running at each given time (including the process that is currently being ran). Generally speaking, on a single core machine, this can be looked at as CPU utilization percentage when multiplied by 100. For example if I had a load-average of 0.50 in the last minute, this means that over the last minute half of the time the CPU was idle as it had no running process. On the other hand if I had load average of 2.50 it means that over the last minute an average of 1.5 process were waiting to their turn to run. so the CPU was overloaded by 150%.

On a multi-core systems things are a bit different, but in order to avoid unnecessary complications one can usually divide the load-average by the number of cores an treat the result as the load average of single core machine. For example let’s say the load average of a two-core machine was 3.00 2.00 0.50. This means that over the last minute we had an average of three runnable process, this means that one process, in average, was queued as there are two core in the machine that can run to process at a time. So the machine was overloaded had a load of 150% its capability. Over the last 5 minutes the load average of 2.00 means that we roughly had 2 process running each time, so the machine was fully utilized but wasn’t overloaded by work. On the other hand over last 15 minutes the load-average of 0.50 means that we could handle 4 time that load without overloading the CPU, we only had (0.50/2)*100=25% CPU utilization in that 15 minutes.

I hope I made the load-average a bit more clearer using the above example. Load-average is an important metric for measuring a system performance, and good understanding of it is beneficial.

Setting Up OmniComplete (Autocompletion) for wxWidgets in Vim

Guy — Sat, 23 Feb 2008 19:27:11 +0000

I use Vim as my main IDE for C/C++ related development (as well as for almost all other development). If you use (or thinking about using) vim as as an IDE, you better get some good autocompletion functionality. This kind of autocompletion is provided by the OmniComplete, which is available since Vim 7.0. Just having the OmniComplete is a nice thing, but it’s much more helpful if configured properly to work with the libraries you use, such as wxWidgets. In this post I will show you how to get working the OmniComplete for wxWidgets, however, the procedure I will show can be easily adapted to almost all libraries.

The first step is to install ctags (if you haven’t got it already installed), this provides the tags file, on which the autocompletion is based. ctags is available directly from the package manager of all popular distro’s.

The next step is to install the OmniCppComplete plugin for vim. The plugin uses the ctags‘ generated tags file for the autocompletion. In order for the autocompletion to properly work with classes, you need to create the ~/.ctags file and add some default options. Each option should be listed in a newline. Your ~/.ctags file should look like this:

--c++-kinds=+p
--fields=+iaS
--extra=+q

These options will now be used as default when running ctags.

The next step is to create a tag file for the wxWidgets library. This can be done using the following command:

 ctags -f ~/.vim/wxwidgetstags -R /usr/include/wx-2.8

Don’t forget to replace /usr/include/wx-2.8 with the path to the header files of wxWidgets. This will create the tags file under your ~/.vim directory.

The next step is to tell vim to use this tags file. This is done by adding the following line to your ~/.vimrc:

:set tags+=~/.vim/wxwidgetstags

And now your done. OmniComplete will now work for wxWidgets, listing members and functions for any of the wxWidgets classes.
Remember you can repeat the last two steps for any other libraries you use in order for the OmniComplete to work with them too.

N.B. don’t forget to run ctags -R . in your projects root dir if you want OmniComplete to work for classes you defined.

Creating Local SVN Repository (Home Repository)

Guy — Mon, 29 Oct 2007 06:39:36 +0000

In this tutorial I will explain how to create a local Subversion (SVN) repository, intended for a single user. I assume that you already know the benefits of keeping track of old revision of projects or important documents such as a resume or a thesis you have been writing. Subversion offers you a very convenient yet strong method to do so, and the easiest way to do so with Subversion (SVN) is to create a local, home, repository intended for a single user – you.

The repository we are going to create will be used by a single user, working locally on the machine. In this tutorial I will assume the repository will be created for a project called “project1″. “project1″ can be a real project you are doing, a paper you are writing or any thing else that can be stored under revision control (that almost everything).

Creating the repository

We will start by creating the repository. From the command line do the following:

$ mkdir /home/user/svnrep
$ cd ~/svnrep
$ svnadmin create project1

In the first line we create a directory to house all of your repositories (I assume you are working under the username “user”). I prefer to use different repositories for different projects that are unrelated, no matter how small they are. Because I use many repositories, I prefer to have a single directory underneath all repositories will reside in an organized way (i.e. no junk files or any other kind of stuff except directories). Next thing, is to cd into the directory we created, and actually create the repository using the svnadmin command. To repository we’ve created is called “project1″.

The brand new “project1″ repository is currently empty and in revision 0. This will change once we put some data in it. The first thing you do with a new repository is to import initial project data into it.

$ svn import /home/user/project1 file:///home/user/svnrep/project1/trunk -m "Initial import of project1"

Will import the current project1′s file into the repository (assuming that project1 indeed resides in /home/user/project1). The trunk appended to the end of the repository URL, is part of the directory layout convention used by many Subversion users. The last part of the command is the message that will be attached to the import in the SVN log.

Now the repository holds data and you are ready to checkout the code from it, and start working.

Checkout a Working Copy and Start Working

When using SVN (as well as in most revision control systems) you don't work directly on the repository. Instead you checkout a working copy from it, and work on this copy. To checkout a working copy from the repository, use:

$ svn co file:///home/user/svnrep/project1/trunk /home/user/project1_work

This will create a working copy of the repository under /home/user/project1_work. You can edit this copy safely.

After you've done editing your working copy, you will want to commit those changes back to the repository. Assuming you are already inside the directory of the working copy, just do:

$ svn commit -m "Some log message"

This will send you changes back to the repository and store them there. Change the "Some log message" to some useful short description of the changes you've made.

Some Useful Commands

To view a list of log messages that were attached to the operations on the repository, use:

$ svn log
$ svn log -r 5:HEAD

The first command will print all log messages. The second command will print the log messages from revision 5 to the latest revision. You can substitute the HEAD with a number to get the log messages of the revisions up to a specific one.

To view the changes you made before committing your code, use:

$ svn diff

Another important command is the one to update your working copy with the latest revision from the repository. This can be done using:

$ svn update

However, since your are the only user of the local repository, you won't have to use this function often (if at all), unless you use two, or more, working copies for your project.

This concludes this tutorial. I hope you know by now how to create and use, a single user local SVN repository.

ssh-keygen Tutorial – Generating RSA and DSA keys

Guy — Fri, 05 Oct 2007 09:32:26 +0000

In this post I will walk you through generating RSA and DSA keys using ssh-keygen. Public key authentication for SSH sessions are far superior to any password authentication and provide much higher security. ssh-keygen is the basic way for generating keys for such kind of authentication. I will also explain how to maintain those keys by changing their associated comments and more importantly by changing the passphrases using this handy utility.

Generating Keys

Generating public keys for authentication is the basic and most often used feature of ssh-keygen. ssh-keygen can generate both RSA and DSA keys. RSA keys have a minimum key length of 768 bits and the default length is 2048. When generating new RSA keys you should use at least 2048 bits of key length unless you really have a good reason for using a shorter and less secure key. The key length for DSA is always 1024 bits as specified in FIPS 186-2. Because DSA key length is limited to 1024, and RSA key length isn’t limited, so one can generate much stronger RSA keys than DSA keys, I prefer using RSA over DSA. Another reason for not using DSA is that DSA is a government standard and one may wonder if the key length was limited deliberately so it will be possible for government agencies to decrypt it.

To generate a pair of public and private keys execute the following command:
ssh-keygen -t rsa -b 2048
You can use “dsa” instead of the “rsa” after the -t to generate a DSA key. The number after the -b specifies the key length in bits.

After executing the command it may take some time to generate the keys (as the program waits for enough entropy to be gathered to generate random numbers). When the key generation is done you would be prompted to enter a filename in which the key will be saved. The public key will have the same filename but it will end with .pub. You should make sure that the key can only be read by you and not by any other user for security reasons.

Next you’ll be prompted to enter a passphrase. Each generated key can be protected by a passphrase. When a key is generated with a passphrase, the key can’t be used without the passphrase, so by using a passphrase one can prevent others from using his private keys without first guessing the passphrase. A good passphrase should be at least 10 characters long. One should stay away from English sentences as their entropy level is just too low to be used as a safe passphrase. I usually use a randomly generated passphrase, as this kind is considered the most secure. If you intend to use the key for accessing a remote machine from inside an automated script you may wish to enter an empty password, so the script won’t need user interaction. In this case just press twice.
If you create a passphrase-less key just make sure you only put it on trusted hosts as it may compromise the remote machine if the key falls to the wrong hands.

After entering you passphrase twice the program will print the key fingerprint, which is some kind of hashing used to distinguish different keys, followed by the default key comment (more on key comments later). After printing the key information the program will terminate. Congratulations, you’ve just created you own public key using ssh-keygen.

Comments

Adding comments to keys can allow you to organize your keys more easily. The comments are stored in end of the public key file and can be viewed in clear text. For example:

cat id_rsa2.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyyA8wePstPC69PeuHFtOwyTecByonsHFAjHbVnZ+h0dpomvLZxUtbknNj3+c7MPYKqKBOx9gUKV/diR/mIDqsb405MlrI1kmNR9zbFGYAAwIH/Gxt0Lv5ffwaqsz7cECHBbMojQGEz3IH3twEvDfF6cu5p00QfP0MSmEi/eB+W+h30NGdqLJCziLDlp409jAfXbQm/4Yx7apLvEmkaYSrb5f/pfvYv1FEV1tS8/J7DgdHUAWo6gyGUUSZJgsyHcuJT7v9Tf0xwiFWOWL9WsWXa9fCKqTeYnYJhHlqfinZRnT/+jkz0OZ7YmXo6j4Hyms3RCOqenIX1W6gnIn+eQIkw== This is the key's comment

As you can see the comment is appended in clear text to the end of the public key file. To alter the comment just edit the public key file with a plain text editor such as nano or vim.
To add a comment to the public key file when generating the key add to the key generation command -C "you comment". For example to generate 4048 bit RSA key with “home machine” as a comment you will do the following:

ssh-keygen -b 4048 -t rsa -C "home machine"

Notice that each copy of a public key can have its own comment and you cannot retrieve the comment from the private key.

Passphrases

Passphrases allow you to prevent unauthorized usage of your key by meaning of protecting the key itself by a password. Although your ssh directory holding the private keys should be unaccessible to other users, the root user of the system, or anyone who can achieve is privileges can access your key and copy it. In this case the passphrase will prevent him from using it.

To add a passphrase to a key just type it when prompted during the key generation process. Keep in mind that the password must be at least 5 characters long. A good passphrase, as I said before, should be at least 10 characters long, and consist of random upper and lower case letters, numbers and symbols.

While the passphrase boosts the security of the key, under some conditions you may want to leave it empty. Leaving the passphrase empty allows you to use the key from within scripts, for example to transfer a file via scp. While passphraseless keys are very useful for scripts just remember to only use them at trusted machines.

You can change the passphrase of key after it’s been created, and you should do it at least annually. To change the passphrase execute:

ssh-keygen -p

After this you will be prompted to enter the location of your private key and enter twice the new passphrase. If you don’t want a passphrase just enter empty one.

End Notes

Feel free to share your public key, as its name suggests, it should be public. Keep in mind that your private key should be kept private. If someone gets hold of your private key, change it imediatly, even if it’s passphrase protected.

Profiling Code Using clock_gettime

Guy — Sat, 22 Sep 2007 08:55:16 +0000

After raising the issue of the low resolution problem of the timer provided by clock() in Resolution Problems in clock(), I’ve ended the post by mentioning to two more functions that should provide high-resolution timers suitable for profiling code. In this post I will discuss one of them, clock_gettime().

The clock_gettime() provides access to several useful timers with the resolution of nanoseconds. First, the prototype for the function is as follows:

int clock_gettime(clockid_t clk_id, struct timespect *tp);

The clk_id allows us to select a specific clock from the several
offered by the system, which includes:

CLOCK_REALTIME, a system-wide realtime clock.
CLOCK_PROCESS_CPUTIME_ID, high-resolution timer provided by the CPU for each process.
CLOCK_THREAD_CPUTIME_ID, high-resolution timer provided by the CPU for each of the threads.

Usually, there are more clocks provided, but I find these three the most useful as they allow to get the execution time spent in the system level, process level and the thread level.

The current clock time, for the chosen clock is stored int the struct provided by the *tp pointer. The timespec struct is defined as follows:

struct timespec {
	time_t tv_sec; /* seconds */
	long tv_nsec; /* nanoseconds */
};

To time the processing time some function took, one should call clock_gettime() twice. Once before the function call and once right after it and subtract the returned timings to get the actual runtime.

Getting the difference between two timespec structs isn’t very complicated and can be acheived using the function diff() defined bellow:

timespec diff(timespec start, timespec end)
{
	timespec temp;
	if ((end.tv_nsec-start.tv_nsec)<0) {
		temp.tv_sec = end.tv_sec-start.tv_sec-1;
		temp.tv_nsec = 1000000000+end.tv_nsec-start.tv_nsec;
	} else {
		temp.tv_sec = end.tv_sec-start.tv_sec;
		temp.tv_nsec = end.tv_nsec-start.tv_nsec;
	}
	return temp;
}

Now let’s move to some real example:

#include 
#include 
using namespace std;
 
timespec diff(timespec start, timespec end);
 
int main()
{
	timespec time1, time2;
	int temp;
	clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time1);
	for (int i = 0; i< 242000000; i++)
		temp+=temp;
	clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time2);
	cout<<diff(time1,time2).tv_sec<<":"<<diff(time1,time2).tv_nsec<<endl;
	return 0;
}
 
timespec diff(timespec start, timespec end)
{
	timespec temp;
	if ((end.tv_nsec-start.tv_nsec)<0) {
		temp.tv_sec = end.tv_sec-start.tv_sec-1;
		temp.tv_nsec = 1000000000+end.tv_nsec-start.tv_nsec;
	} else {
		temp.tv_sec = end.tv_sec-start.tv_sec;
		temp.tv_nsec = end.tv_nsec-start.tv_nsec;
	}
	return temp;
}

To use clock_gettime you need to include time.h and to link to librt.a. If you use gcc just make sure you add -lrt to your list of arguments.

Play a bit with the length of the for loop. As you can see clock_gettime provides much more accurate results and can register very short processing time too. Just remember that as the case with any profiling functions, this function adds a little overhead to your program, so make sure you disable the profiling code, using preprocessor commands for example, in the production release.

26/9/2007 – Update

You may want to take a look at The Revised String Iteration Benchmark post for another, larger, example of using clock_gettime to time performance of code.

Introduction to C++ CGI – Processing Forms

Guy — Fri, 07 Sep 2007 19:11:59 +0000

In this post I will show you how to process HTML forms easily using CGIs in C++. I assume you have already basic knowledge of writing CGIs in C++, if you don’t go a head and read Introduction to C++ CGI.

Processing forms is the basic function of any CGI script and the main purpose of CGIs. As you probably know there are two common ways to send form data back to the web server: “post” and “get”. When form data is sent with the “get” method it is appended to the URL string of the form submission URL. The “post” method is much like the “get” except the data is transmitted via http headers and not via the URL itself. When a form uses “get” it allows the user to easily bookmark the query created by the form as the data is transmitted in URL itself, on the other hand the “post” method allows to send much more data and spares to user from seeing the data in the URL.

Getting the “post” and “get” data is relatively easy. To get the data sent by “get” you can just call getenv("QUERY_STRING") and you will receive a pointer to null-terminated string containing the “get” data. Reading the “post” data is a bit more complicated. The data needs to be read from the standard input, but the program won’t receive an EOF when it reaches the end of the data but instead it should stop reading after reading a specified amount of bytes, which is defined in the environment variable “CONTENT_LENGTH“. So you should read getenv("CONTENT_LENGTH") bytes from the standard input to receive the “post” data.

But here it’s get tricky. The data is received in an encoded way. The data for both “post” and “get” is a string in the form of “var1=value1&var2=value2&var3=value3” and so on, where the “var*” is the names of the submitted form elements and “value*” is their values respectively. And now it gets a bit more tricky, both the elements’ names and values are url-encoded. That means that almost every non alphanumeric character will be encoded as %XX, where XX represents its ASCII value (see the linked Wikipedia article for more information and extra reading). So to sum up the process of processing forms in CGI you should do the following:

Get the data (either by “post” or “get”).
Parse the data to get key-value pairs.
Decode the url-encoded key-value pairs.
Process the data.

Getting the first and last step done is easy. The second and third step require some work, but fortunately when done properly they should only have to be done once, as you will create functions/classes to handle those, so it is really only a one time job. That leaves you only to do yourself the first and last steps.

As I said the second and third steps are troublesome only when not done before. Here is an implementation for those steps that I’ve written as a part of (yet unreleased) CGI library for C++. It implements functions that take the care of the first three steps and return to the user an STL map containing the form elements’ names in the key and their value in the map pair’s value. This functions allows to process forms in a similar way to processing forms in PHP using $_GET and $_POST.

Here is a simple example of processing a “get” form using the above getpost.h header file.

#include 
#include 
#include 
#include "getpost.h"
 
using namespace std;
 
int main()
{
	map<string,string> Get;
	initializeGet(Get); //notice that the variable is passed by reference!
	cout<<"Content-type: text/html"<<endl<<endl;
	cout<<""<<endl;
	cout<<"Processing forms
"<<endl;
	cout<<"\"get\">"<<endl;
	cout<<" \"fname\">First name: "<<endl;
	cout<<" \"text\" name=\"fname\" id=\"fname\">
"<<endl;
	cout<<" \"lname\">Last name: "<<endl;
	cout<<" \"text\" name=\"lname\" id=\"lname\">
"<<endl;
	cout<<" \"submit\" />"<<endl;
	cout<<"

"
<<endl;
 
	if (Get.find("fname")!=Get.end() && Get.find("lname")!=Get.end()) {
		cout<<"Hello "<<Get["fname"]<<" "<<Get["lname"]<<", isn\'t "
			"processing CGI forms with C++ quite easy?"<<endl;
	} else {
		cout<<"Fill up the above from and press submit"<<endl;
	}
	cout<<""<<endl;
 
	return 0;
}

While this is a simple example, it’s enough to demonstarte how stuff is done. The same way you can process the “post” variables. Feel free to take a look at the getpost.h source-code to see how the URL decoding and the other implementation aspects were done.

Update 2009-09-18: There was a bug in getpost.h, see my comment bellow.