Quickly generate lots of random data

What is the quickest way to generate lots of random data on the command line? Usually when I had to wipe hard-drives I would simply use dd to copy from /dev/urandom over the device. However, `/dev/urandom is quite slow and wiping hard-disks can take a long time that way. So, I decided to benchmark a few methods to generate long random streams that are usable in such scenarios.

The benchmark is based on the dd command. For example:

$ dd if=/dev/urandom of=/dev/null bs=4k count=1M

This command will copy a 4GB of random bytes from /dev/urandom over /dev/null. This is probably the simplest method to create a large stream of random bytes, and as it turns out, also the slowest.

The second construct I tried is to use OpenSSL to create a stream of random data which I can read with dd and then write to the target. For example the following would use AES-128 with a random key:

$ openssl rand -hex 32 | openssl enc -aes-128-ctr -in /dev/zero -pass stdin -nosalt | dd if=/dev/stdin of=/dev/null bs=4k count=1M

Let’s breakup this command: openssl rand -hex 32 will generate a random encryption key to be used by the AES encryption. openssl enc -aes-128-ctr -in /dev/zero -pass stdin -nosalt does the actual encryption. It reads the (random) key from stdin and then uses it to encrypt /dev/zero using AES-128 in counter mode. As /dev/zero in an endless stream of zeros, it will simply output an endless stream of (pseudo-)random data. We can also repeat the same command only swapping aes-128-ctr with aes-256-ctr. For most (all?) usage scenarios it doesn’t provided any added security benefits but does have a (small) performance penalty.

Apart from AES, which is a block cipher, we can also try to use actual stream ciphers like the old rc4 and the modern chacha20.

Additionally, many new CPUs come with AES-NI extension which speeds up AES operations considerably. We can repeat the benchmark while disabling AES-NI to see how the different methods will perform if used a CPU that doesn’t support AES-NI.

Finally, I’ve repeated the test with /dev/zero as input, just to have an upper-limit in terms of performance to compare against.

Benchmark results
Benchmark results, time (in seconds) to create 4GB of random data


The results clearly show that you should avoid /dev/urandom. It’s simply not suitable for this task and doesn’t perform well. The various methods of using OpenSSL perform much better. The best performance is achieved by the two AES variants, with aes-128-ctr being the fastest. However, if AES-NI is not supported by the CPU, AES takes a huge performance hit, and is even slower than the (not-so-)good and old RC4. However, ChaCha20 (a modern stream cipher) performs within 30% of AES if AES-NI is available, but if AES-NI is not supported ChaCha20 outperforms the AES variants. So, unless you know AES-NI is supported ChaCha20 is the safe choice.

Getting Started with Let’s Encrypt – Tutorial

A few days ago I got my invitation to Let’s Encrypt Beta Program. For those of you who are not familiar with Let’s encrypt:

Let’s Encrypt is a new free certificate authority, built on a foundation of cooperation and openness, that lets everyone be up and running with basic server certificates for their domains through a simple one-click process.

This short tutorial is intended to get you up and running with your own Let’s Encrypt signed certificates.

The first thing is to get the Let’s Encrypt client:

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt

The main command we will be working with is ./letsencrypt-auto. The first time you will run it, it will also ask for sudo, install various dependencies using your package manager and setup a virtualenv environment.

The next step is to issue the certificate and prove to Let’s Encrypt that you have some control over the domain. The client supports two methods to perform the validation. The first one is the standalone server. It works by setting up a webserver on port 443, and responding to a challenge from the Let’s Encrypt servers. However, if you already have your own web-server running on port 443 (the default for TLS/SSL), you would have to temporarily shut it down. To use the standalone method run:

./letsencrypt-auto --agree-dev-preview --server https://acme-v01.api.letsencrypt.org/directory certonly

The second method is called Webroot authentication. It works by placing a folder (.well-known/acme-challenge) in the document root of your server with files corresponding to responses for challenges.

./letsencrypt-auto --agree-dev-preview --server https://acme-v01.api.letsencrypt.org/directory -a webroot --webroot-path /var/www/html/ certonly

Whatever method you chose, it will ask for a list of domains you want to validate and your email address. You can write multiple domains. The first one will be the Common Name (CN) and the rest will appear in the Subject Alt Name field.

This slideshow requires JavaScript.

The newly generated certificates will be placed in


The important files in this directory are fullchain.pem which contain the full certificate chain to be served to the browser and privkey.pem which is the private key.

An example Nginx configuration will now look like:

        listen 443 ssl;
        ssl_certificate /etc/letsencrypt/live/guyrutenberg.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/guyrutenberg.com/privkey.pem;

Just don’t forget to reload the web-server so configuration changes take effect. No more government snooping on my blog 😉 .


Securing Access using TLS/SSL Client Certificates

This tutorial will guide you in setting up authentication using TLS/SSL Client Certificates. It is a simple one as it would not delve into details about integration with server-side apps. Instead, it simply gives you instructions on how to set up Client Certificate as means to prevent unwanted parties from accessing your website.

For example, one such scenario where it may come useful, is limiting access to sensitive things on the server, like phpMyAdmin. phpMyAdmin for example handles sensitive data (such as database authentication) and does in plain HTTP, which may pose several security risks. Even if the data would be encrypted, someone with access to the application, might find vulnerabilities in it and exploit the relatively high-privileges it got to compromise the server. The solution to this, is also limit who has access to the application at all. A possible solution, which I’ve used, is to limit access to only to the local machine and using SSH or a VPN Tunnel to access phpMyAdmin. A better solution would be to use TLS/SSL Client Certificates. They operate on the connection level and provide both encryption and authentication. They are easier to setup than VPN tunnels and easier to use.

Note that limiting access based on TLS/SSL Client Certificate can only be done on the sub-domain level, because it happens as part of the connection, before any specific HTTP request can be made.

Most of the tutorial is not HTTP server-specific, however the server configuration part relates to Nginx. As other servers (such as Lighttpd) use a very similar configuration for Client Certificates, adapting the instruction should be straightforward.

Creating a CA

The two commands below will create CA private key and a corresponding self-signed certificate for you to sign the TLS client certificates with.

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -aes-128-cbc -out ca.key
openssl req -new -x509 -days 365 -sha256 -key ca.key -out ca.pem

The first command will ask you for a pass phrase for the key. It is used to protect access to the private key. You can decide to not use one by dropping the -aes-128-cbc option from the command.

The second command will ask you to provide some details to be included in the certificate. Those details will be sent to the browser by the web-server to let it know which client certificate to send back when authenticating.

Server Configuration

Upload the ca.pem that was just generated to your server. You should not upload the private key (ca.key).

The following instructions are for Nginx

ssl_client_certificate /path/to/ca.pem;
ssl_verify_client on; # we require client certificates to access

Assuming you already enabled TLS/SSL for the specific sub-domain, your configuration should look something like this:

server {
        server_name subdomain.example.com;

        # SSL configuration
        listen 443 ssl;
        listen [::]:443 ssl;

        ssl_certificate /etc/nginx/example.pem;
        ssl_certificate_key /etc/nginx/example.key;

        ssl_client_certificate /etc/ngingx/ca.pem;
        ssl_verify_client on;

After reloading the server, check that everything is configured correctly by trying to access your site via HTTPS. It should report “400 Bad Request” and say that “No required SSL certificate was sent”.

Creating a Client Certificate

The following commands will create the private key used for the client certificate (client.key) and a corresponding Certificate Signing Request (client.csr) which the owner of the CA certificate can sign (which in the case of this tutorial will be you.

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out client.key
openssl req -new -key client.key -sha256 -out client.csr

You will be asked again to provide some details, this time about you. Those details will be available to server once your browser sends it the client certificate. You can safely leave the “challenge password” empty1.

You can add the flag -aes-128-cbc to the first command if you want the private key for the client certificate to be encrypted. If you opt for it, you will be prompted for a pass phrase just like before.

Signing a Client Certificate

The next step is to sign the certificate signing request from the last step. It is a good practice to overview it and make sure all the details are as expected, so you do not sign anything you would not intend to.

openssl req -in client.csr -text -verify -noout | less

If everything looks just fine, you can sign it with the following command.

openssl x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca.key \
    -set_serial 0x`openssl rand 16 -hex` -sha256 -out client.pem

You will be prompted for your pass phrase for ca.key if you chose one in the first step.

Installing Client Key

Now comes the final part, where we take the signed client certificate, client.pem and combine it with the private key so in can be installed in our browser.

openssl pkcs12 -export -in client.pem -inkey client.key -name "Sub-domain certificate for some name" -out client.p12

Adjust the -name parameter to your liking. It will be used to identify the certificate in various places such as the browser. If your private key was encrypted, you will be prompted to enter a pass phrase for it. Encrytion for certificates in p12 format is mandatory, so you will be prompted to enter a password for the generated file as well. It is OK to reuse the same password here, as those files are practically equivalent. Once imported to you browser, you would not need the password for normal usage, until you would like to import it to another browser.

GlobalSign provides instruction on how to actually install the p12 client certificate for browsers in Linux and Windows.


  1. It is used to “enhance” the security of certificate revocation request, by requiring not only knowledge of the private key, but also the challenge password. Thus, someone who got hold of your private key cannot revoke the certificate by himself. However, this is also the reason why this option is not used more often: When someone steals your private key, usually they will prefer the certificate not to be revoked. 

Default PBKDF2 Iteration Count for Encrypted Keys Generated by OpenSSL

When generating keys with openssl you have the option to encrypt them. It is done by specifying a cipher alogrithm, for example

openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -aes-128-cbc -out key.pem 

generates a 2048 bit RSA key and encrypts it with AES in CBC mode. OpenSSL will prompt you to provide a pass-phrase for the encryption. It is important to understand how that pass-phrase/password will be used to derive a key for the AES encryption. The whole encryption scheme is defined by something called PBES2 1, which in turn uses PBKDF2. The important factor on the computation complexity of PBKDF2, is the number of hash-iterations used.

OpenSSL doesn’t have an option in its command-line utilities to control that number of iterations. However, that number is allowed to change pretty much arbitrarly by the standard, so it is part of the ASN1 representation of the generated encrypted key.

$ openssl asn1parse -i -in key.pem  | head
    0:d=0  hl=4 l=1311 cons: SEQUENCE          
    4:d=1  hl=2 l=  73 cons:  SEQUENCE          
    6:d=2  hl=2 l=   9 prim:   OBJECT            :PBES2
   17:d=2  hl=2 l=  60 cons:   SEQUENCE          
   19:d=3  hl=2 l=  27 cons:    SEQUENCE          
   21:d=4  hl=2 l=   9 prim:     OBJECT            :PBKDF2
   32:d=4  hl=2 l=  14 cons:     SEQUENCE          
   34:d=5  hl=2 l=   8 prim:      OCTET STRING      [HEX DUMP]:F3098873E5AB1A81
   44:d=5  hl=2 l=   2 prim:      INTEGER           :0800
   48:d=3  hl=2 l=  29 cons:    SEQUENCE       

The line saying INTEGER :0800 states the number of iteration used (in hex notation) for the generated key.pem. It means that at least for OpenSSL 1.0.1, the default number of iterations is 0x800=2048. This number is relatively low in modern standards2.

  1. As the name suggest there is also PBES1, which is now obsolete. The main difference is that PBES1 only allowed DES and RC2 to be used as cipers. See RFC 2898 for more details. 
  2. Apple uses 10,000 iterations for iTunes passwords, and LastPass defaults to 5,000 

Extract Public Key from X.509 Certificate as Hex

X.509 certificates are common way to exchange and distribute public key information. For example, most Open Social containers use the OAuth RSA-SHA1 signature method, and distribute their public keys in the X.509 format.

While working on an AppEngine application, I needed to verify requests from such containers. However, there is (currently) no pure python library able of parsing the certificates. This meant that I needed extract the public key out of the certificate manually, and store it in some parsed way inside the Python code.

Fortunately, parsing public keys form a X.509 certificate and representing them as a Hex number turned out simple and easy.
Continue reading Extract Public Key from X.509 Certificate as Hex