Calculate Google Drive Folder Size Using `rclone`

Google Drive lacks a very basic feature: calculating folder size. There is no solution in the web interface to view the total size of a given directory. There are a couple of dubious-looking online “folder size analyzers” that request access permissions to your Google Drive and offer you the basic functionality of calculating folder size. While those apps have a legitimate need for access permissions to your account, you may consider giving those permissions to random apps a questionable decision.

The (very) useful tool rclone, which provides an rsync-like interface to many cloud storage providers, implements this functionality. After configuring rclone to work with your Google Drive, use the size command to determine the size of a given folder:

$ rclone size "gdrive:Pictures/"
Total objects: 1421
Total size: 9.780 GBytes (10501374440 Bytes)

The size is calculated recursively. However, there is no simple way to display the size of each subdirectory recursively.

One Line Implementations of GCD and Extended GCD in Python

Sometimes you need a quick-and-dirty implementation of the greatest common divisor and its extended variant. Actually, because the fractions module comes with a gcd function, the implementation of the extended greatest common divisor algorithm is probably more useful. Both implementations are recursive, but they work nonetheless, even for comparatively large integers.

# simply returns the gcd
gcd = lambda a,b: gcd(b, a % b) if b else a

# egcd(a,b) returns (d,x,y) where d == a*x + b*y
egcd = lambda a,b: (lambda d,x,y: (d, y, x - (a // b) * y))(*egcd(b, a % b)) if b else (a, 1, 0)

The code above is Python 2 and 3 compatible.

`rsync` and FAT File Systems

I’m using rsync to sync files from my computer to a FAT-formatted SD card. Using the --update flag with rsync makes it “skip files that are newer on the receiver.” It seems that it should work as follows: after the first sync, any subsequent syncs will only transfer those files that changed in the meantime. However, I noticed that transfer times are usually longer than expected, which led me to think that things are not working as they should.

As --update relies only on the modification date, obviously something is wrong with it. After ruling out several other possibilities, I guessed that the modification date of some files gets mangled. A quick check of FAT revealed that FAT can only save modification times with 2-second granularity. I’ve also used rsync’s --archive flag, which, among other things, attempts to preserve the modification times of the files it copies. But what about FAT’s lower granularity for modification time? That was apparently the culprit. Some files, when copied, got a modification time that was up to 1 second before the original modification time. Hence, every time I synced, from rsync’s perspective, the target was older than the source and therefore needed to be overwritten.

This can be demonstrated by the following session:
Continue reading `rsync` and FAT File Systems

Let’s Encrypt: Reload Nginx after Renewing Certificates

Today I had an incident that caused my webserver to serve expired certificates. My blog relies on Let’s Encrypt for SSL/TLS certificates, which have to be renewed every 3 months. Usually, the cronjob that runs certbot --renew takes care of it automatically. However, there is one step missing: the server must reload the renewed certificates. Most of the time, the server gets reloaded often enough, so everything is okay, but today, it’s been quite a while since the last time the nginx server was restarted, so expired certificates were served and the blog became unavailable.

To work around it, we can make sure nginx reloads its configuration after each successful certificate renewal. The automatic renewal is defined in /etc/cron.d/certbot. The default contents under Debian Jessie are as follows:

# /etc/cron.d/certbot: crontab entries for the certbot package
#
# Upstream recommends attempting renewal twice a day
#
# Eventually, this will be an opportunity to validate certificates
# haven't been revoked, etc.  Renewal will only occur if expiration
# is within 30 days.
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

0 */12 * * * root test -x /usr/bin/certbot && perl -e 'sleep int(rand(3600))' && certbot -q renew

The last line makes sure certificate renewal runs twice a day. Append --renew-hook "/etc/init.d/nginx reload" to it, so it looks like this:

0 */12 * * * root test -x /usr/bin/certbot && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "/etc/init.d/nginx reload"

The --renew-hook runs the next argument after each successful certificate renewal. In our case, we use it to reload the nginx configuration, which also reloads the newly renewed certificates.

Update 2019-02-27:

renew-hook has been deprecated in recent versions of certbot. Plus, Debian moved from using cronjobs for automatic renewals to a systemd timer if they are available. On the other hand, now certbot supports having hooks in configuration files. So, instead of what is described above, I would suggest creating a file /etc/letsencrypt/renewal-hooks/deploy/01-reload-nginx with the following content:

#! /bin/sh
set -e

/etc/init.d/nginx configtest
/etc/init.d/nginx reload

don’t forget to make the file executable.

LyX: Missing Title When Passing `ignorenonframetext` to Beamer

Passing ignorenonframetext as an option to Beamer causes it to ignore all text that is not inside a frame. It is useful when you want to add content for the article version of the presentation (or simply script lines for yourself) that would not show in the regular presentation. LyX puts the title elements outside any frame. Therefore, if you use ignorenonframetext, you end up missing the title frame. The solution is to manually wrap the title block (the title, author, institute, etc.) in a frame and append maketitle to it. This will cause the title frame to be rendered correctly.

Optimize LaTeX PDF Output for Kindle

Kindle can display PDFs, but usually the result is very hard to read. Normal PDFs are not suitable, especially when it comes to paper size for the relatively small display of the Kindle. For a forthcoming project, which I intend to write in LaTeX and read on Kindle, I looked into optimizing the document settings so the result would be rendered in a readable manner on Kindle.

I’ve started with the normal article class. The result is not good at all:
article While Kindle zooms in automatically to remove the usually very wide margins LaTeX uses, the big (A4) paper size still results in a tiny font on the Kindle display. Switching to KOMA-Script is a bit better, but mainly provides better mechanisms to control the paper size for later experiments.
scrartcl

The next try is simply to use the A5 paper size. The result is getting better, but the paper size is still too big. Setting the paper size manually to 12cm by 9cm (the screen’s physical dimensions) and setting the pagestyle to empty (which removes the page numbering, among other things) results in much better results, but because of the (still) wide margins and the auto-zoom, the font size is too big and not enough content fits on a page:
scrartcl_12x9

Finally, manually setting the text area to be a bit smaller (11cm by 8cm) than the paper size results in small margins and very little auto-zoom. The output can be clearly read on the Kindle, and still quite a bit of text fits on a single page:


The LaTeX code for the last example is:

documentclass[DIV=calc,paper=9cm:12cm,pagesize]{scrartcl}

areaset{8cm}{11cm}
pagestyle{empty}
usepackage{lipsum}
begin{document}
lipsum
end{document}

KOMA-Script: Specifying Binding Correction for RTL Documents

The KOMA-Script bundle provides an option to specify the amount of binding correction needed to compensate for the width lost in the binding process. By default, it is added to the left margin, which is where the binding is applied for left-to-right languages. However, if a document is written in Hebrew or Arabic, it is bound on the right. The KOMA-Script manual does not consider that option. After a bit of playing around, I’ve found that simply using a negative value for the binding correction works.

For example, if in an English document you would use

documentclass[BCOR=8.25mm]{scrreprt}

For Hebrew you would set

documentclass[BCOR=-8.25mm]{scrreprt}

Installing OrderNet Pro on Debian Jessie

OrderNet is a popular stock trading platform in Israel. OrderNet comes in two versions: the regular version is based on Silverlight and can be used on Linux using Pipelight. The Pro version is a desktop program written in .NET Framework and features a better interface. This post walks through the steps needed to get OrderNet Pro running on Debian Jessie using Wine.

Continue reading Installing OrderNet Pro on Debian Jessie