Author: Guy

Moving Debian to a New Computer

These are the steps I took to migrate a Debian installation from an old computer to a new one. I took out the old SSD, and connected it via an external enclosure to the new computer, and booted via a live USB.

The next step is to copy over the entire disk from the old SSD to the new one. Because we will copy everything, even the partitions’ UUIDs will remain the same and no extra steps should be necessary apart from adjusting some partition sizes. Be very careful with the output and input devices. In my case the old SSD is connected as the external drive /dev/sdb and the new one is /dev/nvme0n1.

$ sudo dd if=/dev/sdb of=/dev/nvme0n1 bs=4K status=progress

Refresh the partition table:

$ sudo partprobe

Grow /dev/nvme0n1p3to fill the entire partition using gparted.

$ sudo cryptsetup --token-only open /dev/nvme0n1p3 new-root
$ sudo cryptsetup resize --token-only new-root

(you can omit --token-only if you don’t use a Yubikey to unlock the drive).

Mount the btrfs root file system and resize it:

$ sudo mount -t btrfs /dev/mapper/new-root /mnt
$ sudo btrfs filesystem resize max /mnt

Now you are ready to reboot into the new system.

Reencrypt the LUKS partition

Moving to a new SSD is also a good opportunity to rotate the master key of the LUKS encrypted root partition. This can be done while the disk is online and mounted, and takes some time.

The reencryption implementation doesn’t properly support FIDO2 keys for unlocking. We would have to delete those and re-register the keys afterwards. Select a key slot with a passphrase and pass it using the --key-slot parameter. You can check which key-slot is in use using cryptsetup luksDump

$ sudo cryptsetup reencrypt /dev/nvme0n1p3 --key-slot 1

Once done, re-enroll any FIDO2 keys you have by running the following command for each key:

$ sudo systemd-cryptenroll /dev/nvme0n1p3 --fido2-device=auto  --fido2-with-client-pin=yes

Enabling Secure Boot

Initially, I had problems with Secure Boot refusing to boot the new installation. They were resolved by reinstalling shim-signed and grub-efi-amd64-signed. Additionally, I had to enable “Allow Microsoft 3rd Party UEFI CA” in the Secure Boot settings of the UEFI:

Lenovo T14 Gen 4 Secure Boot settings

Creating a WireGuard profile for Cloudflare Warp

Connecting to Cloudflare Warp directly via wg can have advantages in flexibility or specific scenarios. For example, the Warp client, warp-cli would refuse to establish connection if it can’t override /etc/resolve.conf. By connecting directly using WireGuard, you get control over all that.

The first step is to install warp-cli and register using warp-cli register. This will create the WireGuard private-key used for the connection and register it with Cloudflare. The private key can be found in /var/lib/cloudflare-warp/reg.json. The endpoint data and Cloudflare’s public key should be constant. Alternative endpoints are listed in /var/lib/cloudflare-warp/conf.json.

An easy way to read the json configuration files is using jq:

$ sudo jq . /var/lib/cludflare-warp-/conf.json

Adjust the following template accordingly, and put in int /etc/wireguard/warp.conf:

[Interface]
PrivateKey = XXXXXXXXXXXX  
Address = 172.16.0.2/32
Address = 2606:4700:110:892f:607d:85a6:5e07:70cf/128
[Peer]
PublicKey = bmXOC+F1FxEMF9dyiK2H5/1SUtzH0JuVo51h2wPfgyo=
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = engage.cloudflareclient.com:2408

You can start the tunnel using

$ sudo wg-quick up warp`

Alternatively, you can import it to NetworkManager and be able to easily start it from the Gnome Quick Settings.

$ sudo nmcli connection import type wireguard file /etc/wireguard/warp.conf

You can easily check that the tunnel works, by visiting https://www.cloudflare.com/cdn-cgi/trace/ and looking for the line that says warp=on.

Sometimes, IPv4 won’t work while IPv6 works. Restarting the VPN several times can resolve the issue.

while ! ping -w1 -c1 1.1.1.1; do wg-quick down wgcf-profile; wg-quick up wgcf-profile; done

or using nmcli:

while ! ping -w1 -c1 1.1.1.1; do nmcli connection down wgcf-profile; nmcli connection up wgcf-profile; done

Disabling the Cloudflare client

The Cloudflare client might interfere with the Wireguard profile. It’s best to didable it:

$ sudo systemctl disable --now  warp-svc.service
$ systemctl --user disable --now warp-taskbar.service

Install GNOME 44 on Debian Unstable

GNOME 44 recently got released. Because Debian Bookworm is undergoing a freeze, GNOME 44 is not yet available in Debian Unstable. It’s very similar to the GNOME 40 situation 2 years ago. While the wait for the freeze to be over can take a long time, Debian already has (most) of the updated pacakges in the experimental, and we can update to GNOME 44 through it:

$ sudo apt install -t experimental baobab eog evince gdm3 gjs gnome-backgrounds gnome-calculator gnome-characters gnome-contacts gnome-control-center gnome-disk-utility gnome-font-viewer gnome-keyring gnome-logs gnome-menus gnome-online-accounts gnome-remote-desktop gnome-session gnome-settings-daemon gnome-shell gnome-shell-extensions gnome-software gnome-system-monitor gnome-text-editor gnome-user-docs mutter gnome-desktop3-data
$ sudo apt-mark auto baobab eog evince gdm3 gjs gnome-backgrounds gnome-calculator gnome-characters gnome-contacts gnome-control-center gnome-disk-utility gnome-font-viewer gnome-keyring gnome-logs gnome-menus gnome-online-accounts gnome-remote-desktop gnome-session gnome-settings-daemon gnome-shell gnome-shell-extensions gnome-software gnome-system-monitor gnome-text-editor gnome-user-docs mutter gnome-desktop3-data

Protecting OpenSSL private keys using PKCS#8

When generating private keys with OpenSSL, by default they are unprotected by a passphrase. For example:

$ openssl genrsa
-----BEGIN PRIVATE KEY----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCY+q1kOPM4RF5T
...
Rjet4T2TrJmFzIL1dsgJACU=
-----END PRIVATE KEY-----

generates a plaintext private key. By using the -aes256 parameters, the generated key is protected according to PKCS#8:

$ openssl genrsa -aes256
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFLTBXBgkqhkiG9w0BBQ0wSjApBgkqhkiG9w0BBQwwHAQITQpYPEgrGN0CAggA
...
XIdi3hSPE8NoPWnROVGMnGbFOMm36g6064nZzQBD4r+4
-----END ENCRYPTED PRIVATE KEY-----

However, is it actually secure?

The default protection used by genrsa is PBKDF2 with 2048 (0x800) iterations of HMAC-SHA256. This can be verified by examining the PKCS#8 key, which is ASN.1 encoded.

$ openssl asn1parse -in key.priv 
    0:d=0  hl=4 l=1325 cons: SEQUENCE          
    4:d=1  hl=2 l=  87 cons: SEQUENCE          
    6:d=2  hl=2 l=   9 prim: OBJECT            :PBES2
   17:d=2  hl=2 l=  74 cons: SEQUENCE          
   19:d=3  hl=2 l=  41 cons: SEQUENCE          
   21:d=4  hl=2 l=   9 prim: OBJECT            :PBKDF2
   32:d=4  hl=2 l=  28 cons: SEQUENCE          
   34:d=5  hl=2 l=   8 prim: OCTET STRING      [HEX DUMP]:EAD8B97C8EAD1414
   44:d=5  hl=2 l=   2 prim: INTEGER           :0800
   48:d=5  hl=2 l=  12 cons: SEQUENCE          
   50:d=6  hl=2 l=   8 prim: OBJECT            :hmacWithSHA256
   60:d=6  hl=2 l=   0 prim: NULL              
   62:d=3  hl=2 l=  29 cons: SEQUENCE          
   64:d=4  hl=2 l=   9 prim: OBJECT            :aes-256-cbc
   75:d=4  hl=2 l=  16 prim: OCTET STRING      [HEX DUMP]:290DB8F1786A9C8667829A4A394A8FB7
   93:d=1  hl=4 l=1232 prim: OCTET STRING      [HEX DUMP]:D828C11F170D22B82801A567A9136AA293D630692B88C1F07987ED256C29DC45C4625709A0D36039B22E5A8BCA2B400C590C2560CED3629D1F8C5D77AD...CC5F8027A3A5ED533A00D626E7DDC4ABC85547

A meager 2048 SHA256 iterations provide very little added protection on top of the password intrinsic entropy.

Better protection can be achieved by using scrypt to protect the PKCS#8 encoded key. You can upgrade an existing key’s protection to scrypt:

openssl pkcs8 -in key.priv -topk8 -scrypt -out key-scrypt.priv

The command will prompt for the key’s passphrase and re-encrypt it using a key derived from the same passphrase using scrypt.

openssl pkcs8 -in key3.priv -topk8 -scrypt -out key5priv

The default openssl scrypt parameters, N=0x4000=2^14, r=8, p=1, are simply weak. They are what scrypt’s author recommended to achieve 100ms per iteration on his 2002-era laptop. While better than the PBKDF2 protection offered by OpenSSL, you might want stronger protection.

Choosing better scrypt parameters

Before we dive into the parameters, let’s understand the scrypt algorithm briefly. Scrypt takes three input parameters: password, salt, and cost parameters. The password and salt are user-provided inputs, whereas the cost parameters are fixed for a given implementation.

The cost parameters determine the computational cost of the algorithm. Higher cost parameters result in increased computational requirements, making it harder to brute-force the derived key. The cost parameters are as follows:

  • N: This is the CPU/memory cost parameter. It determines the number of iterations the algorithm performs. The value of N must be a power of two.
  • r: This is the block size parameter. It determines the size of the blocks of memory used during the algorithm’s execution.
  • p: This is the parallelization parameter. Doesn’t affect the memory consumption of the algorithm. However, despite it’s name, OpenSSL doesn’t perform any parallelization, instead opting to run the algorithm serially.

The original article recommends r=8 and p=1 as a good ratio between memory and CPU costs. This allows us to scale N almost arbitrary, increasing both CPU and memory costs. However, OpenSSL, by default, limits it’s memory usage.

$ openssl pkcs8 -in key.priv -topk8 -scrypt -scrypt_N 0x8000 -out key-scrypt2.priv
Enter pass phrase for key.priv:
Error setting PBE algorithm
40E778DD277F0000:error:030000AC:digital envelope routines:scrypt_alg:memory limit exceeded:../providers/implementations/kdfs/scrypt.c:482:
40E778DD277F0000:error:068000E3:asn1 encoding routines:PKCS5_pbe2_set_scrypt:invalid scrypt parameters:../crypto/asn1/p5_scrypt.c:59:

This limitation severely our parameters choice. The only option to increase security, is through the CPU cost parameter (p). We can set it (almost) arbitrary large, getting added security. For example:

$ openssl pkcs8 -in MOK.priv -topk8 -scrypt -scrypt_p 32 -passin pass:1234 -passout pass:1234 -out key-scrypt3.priv

Split DNS using systemd-resolved

Many corporate environments have internal DNS servers that are required to resolve internal resources. However, you might prefer a different DNS server for external resources, for example 1.1.1.1 or 8.8.8.8. This allows you to use more secure DNS features like DNS over TLS (DoT). The solution is to set up systemd-resolved as your DNS resolver, and configure it for split DNS resolving.

Starting with systemd 251, Debian ships systemd-resolved as a separate package. If it isn’t installed, go ahead and install it.

$ sudo apt install systemd-resolved
$ sudo systemctl enable --now systemd-resolved.service

Create the following configuration file under /etc/systemd/resolved.conf.d/99-split.conf:

[Resolve]
DNS=1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com

Domains=~.
DNSOverTLS=opportunistic

Domains=~. gives priority to the global DNS (1.1.1.1 in our case) over the link-local DNS configurations which are pushed through DHCP (like internal DNS servers).

DNSOverTLS=opportunistic defaults to DNS over TLS but allows fallback to regular DNS. This is useful when corporate DNS doesn’t support DNS over TLS and you still want to resolve corporate internal domains.

Restart systemd-resolved to reload the configuration:

$ sudo systemctl restart systemd-resolved

The final step is to redirect programs relying on /etc/resolv.conf (possibly through the glibc API) to the systemd-resolved resolver. The recommended way according to the systemd-resolved man page is to symlink it to /run/systemd/resolv/stup-resolv.conf.

$ sudo ln -rsf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf

F5 VPN

F5 VPN doesn’t play well with the above configuration. First, F5 VPN tries to overwrite the DNS configuration in /etc/resolv.conf, by removing the existing file and replacing it with its own (pushing corporate DNS server configuration through it). The solution is to prevent F5 VPN from deleting the /etc/resolv.conf, by setting it to immutable. However, we cannot chattr +i a symlink. We have to resort to copying the configuration statically, and then protect it.

$ sudo cp /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf
$ sudo chattr +i /etc/resolv.conf

Finally, because now F5 VPN can’t update the DNS configuration, we would have to manually configure the corporate DNS servers and the search domains.

$ sudo resolvectl dns tun0 192.168.100.20 192.168.100.22
$ sudo resolvectl domain tun0 ~example.corp ~example.local

Symbolized stacktraces for a package in Debian

By default, Debian packages aren’t symbolized, resulting in unreadable stacktraces:

#0  0x00007fb9d7a3a774 in ?? ()
#1  0x00005574a4450ea0 in ?? ()
#2  0x00005574a42cea60 in ?? ()
#3  0x00005574a3f8bd20 in ?? ()
#4  0x00007ffe0d782200 in ?? ()

The first step is to determine the right package containing the debug symbols for your binary. This can be done using find-dbgsym-packages from the debian-goodies packages:

$ find-dbgsym-packages /usr/bin/gnome-control-center

Install the relevant *-dbgsym packages, for example:

$ sudo apt install gnome-control-center-dbgsym libglib2.0-0-dbgsym

And now you can have symbolized stacktrace:

#0  0x00007fb9d7a3a774 in g_type_check_instance_cast (type_instance=0x100000002, iface_type=93959409689392) at ../../../gobject/gtype.c:4122
#1  0x00005574a0d1382f in private_key_picker_helper (self=self@entry=0x5574a3f8bd20, filename=filename@entry=0x5574a42cea60 "*******************************************", changed=changed@entry=1)
    at ../panels/network/wireless-security/eap-method-tls.c:252
#2  0x00005574a0d13a34 in private_key_picker_file_set_cb (chooser=<optimized out>, user_data=0x5574a3f8bd20) at ../panels/network/wireless-security/eap-method-tls.c:297
#3  0x00007fb9d7a173b0 in g_closure_invoke (closure=0x5574a4302fb0, return_value=return_value@entry=0x0, n_param_values=2, param_values=param_values@entry=0x7ffe0d782200, invocation_hint=invocation_hint@entry=0x7ffe0d782180)
    at ../../../gobject/gclosure.c:832
#4  0x00007fb9d7a2a076 in signal_emit_unlocked_R (node=node@entry=0x5574a1335ba0, detail=detail@entry=1327, instance=instance@entry=0x5574a3f91350, emission_return=emission_return@entry=0x0, instance_and_params=instance_and_params@entry=0x7ffe0d782200)
    at ../../../gobject/gsignal.c:3796
#5  0x00007fb9d7a30bf5 in g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>, var_args=var_args@entry=0x7ffe0d7823a0) at ../../../gobject/gsignal.c:3549

Further notes

By default, Debian doesn’t create core dumps. This can be changed (for the current running terminal session) with

$ ulimit -c unlimited

You can create more sensible core dump names using:

sudo sysctl -w kernel.core_pattern=/tmp/core-%e.%p.%h.%t

English interface on the Xiaomi AIoT AX3600 Router

Three years ago, I bought the Xiaomi AIoT AX3600 Router. Back at the time, only the Chinese version was available, and that one only supported Chinese as an interface language for the admin panel, which isn’t great. Time passed, and the international variant came out. Both versions have different firmwares, so the Chinese version remained pegged to the Chinese interface. Luckily, someone over at OpenWRT found out that you can install the internation firmware on the Chinese variant. The firmware is available over here.

$ sha256sum miwifi_r3600_all_6510e_3.0.22_INT.bin
67881cea85f8452bb63b3067d3100796a9c1b4f65aaa3479dcb4d01216bc6ce4  ./miwifi_r3600_all_6510e_3.0.22_INT.bin

Once you install it, you’ll have an option to change the language to English.

ls colors broken under Solarized dark theme

A recent change introduced in GNU coreutils changed the default dircolors for backup files to make them less conspicuous. However, despite having stated that it works on dark backgrounds, this change made it impossible to see backup files such as .tar, .swp, .bak, .old when using the dark variant of the Solarized color scheme of the terminal. It can be seen in the following screenshots:

Solarized dark
Solarized light

To fix it, we’ll override the colors by creating ~/.dircolors file:

$ dircolors -p | sed "s/00;90/00;30/g" > ~/.dircolors
$ eval $(dircolors -b ~/.dircolors)

This will set the color of backup files to black, which makes them not stand out, but still readable.

This is the bash function I used to pretty-print all ls colors:

( # Run in a subshell so it won't crash current color settings 
    dircolors -b >/dev/null
    IFS=:
    for ls_color in ${LS_COLORS[@]}; do # For all colors
        color=${ls_color##*=}
        ext=${ls_color%%=*}
        echo -en "\E[${color}m${ext}\E[0m " # echo color and extension
    done
    echo
)

Another option, albeit more verbose, would be

$ dircolors --print-ls-colors ~/.dircolors | paste -sd ''

Tarsum in Rust

Almost 14 years ago, I wrote a [small utility, named tarsum, to calculate checksums on files inside a tar archive. It was useful for verifying data inside backups. Recently, I decided to rewrite it in Rust. It’s available from https://github.com/guyru/tarsum.

Installation using cargo is straight forward:

$ cargo install --git https://github.com/guyru/tarsum

Surprisingly, testing on a large tar archive (recent Linux tarball, 1.3 GB), the performance of both Python and Rust implementation is very similar.