Display reboot required message on Debian

You can use MOTD (message of the day) to let you know if a Debian server requires reboot and why upon login.

Create a new file named /etc/update-motd.d/98-reboot-required and add to it the following lines:

#!/bin/sh -e
#
# helper for update-motd

if [ -f /var/run/reboot-required ]; then
	echo "*** System restart required ***"
        cat /var/run/reboot-required.pkgs
fi

Make the file executable:

$ sudo chmod +x /etc/update-motd.d/98-reboot-required

Now, you can test the new MOTD script using:

$ run-parts --lsbsysinit /etc/update-motd.d

If you have any installed updates that require reboot, you will get a message stating so, with a list of the packages that require the reboot.

*** System restart required ***
linux-image-5.10.0-19-cloud-amd64

Signing kernel modules for Secure Boot

Some time ago, I needed to use the v4l2loopback module. It can be installed via:

$ sudo apt install v4l2loopback-dkms

Normally, after installing a module, you can just modprobe it, and it will load. However, due to Secure Boot, it will fail.

$ sudo modprobe v4l2loopback 
modprobe: ERROR: could not insert 'v4l2loopback': Operation not permitted

The problem is that the v4l2loopback isn’t signed. For example, compare the output of:

$ /usr/sbin/modinfo -F signer v4l2loopback

which is empty, versus

$ /usr/sbin/modinfo -F signer xor
Debian Secure Boot CA

The solution would be to sign the v4l2loopback module ourselves.

Creating a key

The update-secureboot-policy script available in Ubuntu’s shim-signed package is able to generate Machine Owner Keys (MOK) by itself. However, the currently available in Debian Unstable doesn’t have the key generation functionality. We can either fetch the Ubuntu version or generate the keys ourselves.

$ wget https://git.launchpad.net/ubuntu/+source/shim-signed/plain/update-secureboot-policy
$ chmod +x ./update-secureboot-policy
$ sudo ./update-secureboot-policy --new-key

Or through generating the keys ourselves:

$ sudo mkdir -p /var/lib/shim-signed/mok
$ cd /var/lib/shim-signed/mok/
$ sudo openssl genrsa -aes256 -out MOK.priv
$ sudo openssl req \
        -subj "/CN=`hostname -s | cut -b1-31` Secure Boot Module Signature key" \
        -new -x509 -nodes -days 36500 -outform DER \
        -key MOK.priv \
        -out MOK.der

Write down the passphrase for your private key. You will need it whenever you want to sign drivers.

Now we enroll the newly created key:

$ sudo mokutil --import MOK.der

You will be prompted for a password. This password will be required after reboot in order to complete the key enrollment, you will not need it afterwards.

After reboot, check that your key was indeed enrolled:

$ mokutil --list-enrolled

Signing the module

We need to put the passphrase for the private key in the KBUILD_SIGN_PIN env variable:

$ read -s KBUILD_SIGN_PIN
$ export KBUILD_SIGN_PIN

Now we can do the actual signing:

$ cd /usr/lib/modules/$(uname -r)/updates/dkms
$ sudo --preserve-env=KBUILD_SIGN_PIN /usr/lib/linux-kbuild-$(uname -r | cut -d. -f1-2)/scripts/sign-file sha256 /var/lib/shim-signed/mok/MOK{.priv,.der} v4l2loopback.ko

You will need to repeat this step for every new kernel that you install.

Setting up WireGuard on Debian

WireGuard is a modern VPN solution, which is much easier to configure than OpenVPN and its likes. In this tutorial, we assume a simple setup where we want to route all clients network traffic through the VPN, exiting through the server.

Server configuration

$ sudo apt install wireguard
$ PRIVATE_KEY=$(wg genkey)

Now we create the configuration file for our tunnel (wg0).

$ cat <<EOF | sudo tee /etc/wireguard/wg0.conf
[Interface]
PrivateKey = $PRIVATE_KEY
Address = 10.8.0.1/24, fd0d:d325:45c0::1/64
ListenPort = 51820
SaveConfig = true

PostUp = iptables -t nat -I POSTROUTING -o eth0 -j MASQUERADE
PostUp = ip6tables -t nat -I POSTROUTING -o eth0 -j MASQUERADE
PreDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
PreDown = ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
EOF

If you have a firewall, you’ll need to open up UDP port 51820 (or whatever configured as the ListenPort).

Enable IPv4 and IPv6 forwarding, so we can route all the client traffic through the server (and reload sysctl configuration)

$ echo net.ipv4.ip_forward=1 | sudo tee /etc/sysctl.d/40-wireguard.conf
$ echo net.ipv6.conf.all.forwarding=1 | sudo tee -a /etc/sysctl.d/40-wireguard.conf
$ sudo sysctl --system 

Enable and start the WireGuard service:

$ sudo systemctl enable --now wg-quick@wg0.service

Finally, take note of the server public key (it will be a short base64 encoded string):

$ echo $PRIVATE_KEY | wg pubkey

You’ll need it in the next step.

Peer configuration

$ sudo apt install wireguard
$ PRIVATE_KEY=$(wg genkey)
$ PUBLIC_KEY=$(echo $PRIVATE_KEY | wg pubkey)
$ SERVER_PUBLIC_KEY=6TDw+U2WFhkaKUy/xXrCRtuZvB2m2SFN7URZA5AkGis=
$ SERVER_IP=8.8.8.8

Replace the value of SERVER_PUBLIC_KEY with the public key of your server, and SERVER_IP with the correct IP address of your server.

Edit /etc/wireguard/wg0.conf

$ cat <<EOF | sudo tee /etc/wireguard/wg0.conf
[Interface]
PrivateKey = $PRIVATE_KEY
Address = 10.8.0.2/24, fd0d:d325:45c0::2/64

[Peer]
PublicKey = $SERVER_PUBLIC_KEY
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = $SERVER_IP:51820
EOF

Setting AllowedIPs = 0.0.0.0/0, ::/0 will route all traffic through the VPN connection. If you don’t want to do that, edit the configuration file, and set AllowedIPs = 10.8.0.0/24, fd0d:d325:45c0::0/64.

We need to make the server aware of the peer. The following command should be executed on the server.

$ sudo wg set wg0 peer $PUBLIC_KEY allowed-ips 10.8.0.2,fd0d:d325:45c0::2

where PUBLIC_KEY is the value of the client’s public key (stored in the $PUBLIC_KEY environment variable).

The /etc/wireguard/wg0.conf will be updated according and make the added peer configuration persistent due to the SaveConfig = true. The configuration update will take place the next time the WireGuard interface goes down.

You can also import the profile into NetworkManager and easily enable/disable the new interface via GNOME:

$ sudo nmcli connection import type wireguard file /etc/wireguard/wg0.conf

Android Peer

It is also useful to have WireGuard on the phone. WireGuard supports both iOS and Android, and the setup should be similar in both cases. Start by installing WireGuard from the Play Store. The next step is to generate the required configuration. It can be done directly on the phone, or by creating a configuration file on your computer and transferring it, which I find simpler.

$ PRIVATE_KEY=$(wg genkey)
$ PUBLIC_KEY=$(echo $PRIVATE_KEY | wg pubkey)
$ SERVER_PUBLIC_KEY=6TDw+U2WFhkaKUy/xXrCRtuZvB2m2SFN7URZA5AkGis=
$ SERVER_IP=8.8.8.8
$ cat <<EOF > wg0.conf
[Interface]
PrivateKey = $PRIVATE_KEY
Address = 10.8.0.3/24, fd0d:d325:45c0::3/64

[Peer]
PublicKey = $SERVER_PUBLIC_KEY
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = $SERVER_IP:51820
EOF

Again, make the server aware of the client by running the following command on the server:

$ sudo wg set wg0 peer $PUBLIC_KEY allowed-ips 10.8.0.3,fd0d:d325:45c0::3

Transfer the configuration file wg0.conf to the phone and load it using the WireGuard app.

Update 2023-11-30: Added IPv6 configuration.

GIO can’t mount SMB

When trying to mount SMB share using gio you might encounter the following error:

$ gio mount smb://ptnas1.cellebrite.local
gio: smb://nas.corp.local/: Location is not mountable

This error might be due to a missing gvfs backend. The smb backend should be located in /usr/share/gvfs/mounts/smb.mount. If it is missing, you should install the gvfs-backends package and it should resolve the mounting issue.

If the mount was successful, but you can’t see it under $XDG_SESSION_DESKTOP/gvfss/, you are probably missing the FUSE server that makes the mounted filesystem available to all applications and not only to GIO aware applications. You can install the FUSE server by installing the gvfs-fuse package. The server will automatically run after reboot. If you want to start the server immediately, you can start it manually:

$ /usr/lib/gvfs/gvfsd-fuse /run/user/1000/gvfs

List obsolete packages using Apt

Apt 2.0 introduced a new feature allowing to use smarter patterns when listing packages. This allows to list all obsolete packages using the specifier ?obsolete or the shorthand ~o.

$ apt list ?obsolete

It will list all locally installed packages and packages.

You can also list all packages not provided by Debian (it will also catch packages once provided by Debian but since then remove from the repositories):

$ apt list --installed "?not(?origin(debian))"

Slack screen sharing under Wayland

To get screen sharing in Slack to properly work under Wayland, you need to explicitly tell Slack to use PipeWire for screen capturing. Copy Slack’s desktop file from /usr/share/applications/slack.desktop to ~/.local/share/applications/slack.desktop and modify the Exec line to look like:

Exec=/usr/bin/slack --enable-features=WebRTCPipeWireCapturer %U

Now restart Slack and screen sharing should work properly.

Unlock LUKS volume with a YubiKey

Update: The dracut configuration has been updated and now udev consistently recognizes the YubiKey in the initramfs.

Unlocking LUKS encrypted drives with a YubiKey has been supported since systemd 248. In Debian, systemd>=250 is required, as the feature has not been enabled in prior versions. This tutorial is geared towards Yubikeys, but it should work with slight modifications with any other FIDO2 token.

YubiKey series 5 and later should support the hmac-secret extension. You can make sure your Yubikey supports the needed hmac-secret extension by querying it with ykman:

$ ykman --diagnose 2>&1 | grep hmac-secret

Backup your LUKS header

In case you mess anything up, you would need a backup of your LUKS header. Remember to save your backup to some external storage, so you can actually access it if anything goes sideways.

# cryptsetup luksHeaderBackup /dev/nvme0n1p3 --header-backup-file /media/guyru/E474-2D80/luks_backup.bin

Set FIDO2 PIN

We would like to set a FIDO2 PIN for the Yubikey, so unlocking the encrypted drive would require both the physical Yubikey and the PIN. You can set the PIN using:

$ ykman fido access change-pin

Enroll the Yubikey

Start by verifying that systemd-cryptenroll can see and can use your YubiKey:

$ systemd-cryptenroll --fido2-device=list
PATH         MANUFACTURER PRODUCT
/dev/hidraw0 Yubico       YubiKey FIDO+CCID

Now, enroll the Yubikey, replacing /dev/nvme0n1p3 with the block device of the LUKS encrypted drive.

$ sudo systemd-cryptenroll /dev/nvme0n1p3 --fido2-device=auto  --fido2-with-client-pin=yes
🔐 Please enter current passphrase for disk /dev/nvme0n1p3: (no echo)
Initializing FIDO2 credential on security token.
👆 (Hint: This might require confirmation of user presence on security token.)
🔐 Please enter security token PIN: (no echo)
Generating secret key on FIDO2 security token.
👆 In order to allow secret key generation, please confirm presence on security token.
New FIDO2 token enrolled as key slot 0.

Modify /etc/crypttab

We need to modify /etc/crypttab in order to tell cryptsetup to unlock the device using the YubiKey. Add fido2-device=auto in the options field of the crypttab entry for your device. For example:

nvme0n1p3_crypt UUID=307a6bef-5599-4963-8ce0-d9e999026c1a none luks,discard,fido2-device=auto

Switch to dracut

Debian’s default initramfs generator, update-initramfs of the initramfs-tools is using the old cryptsetup for mounting encrypted drives. However, cryptsetup doesn’t recognize the fido2-device option. Running update-initramfs will fail with the following error:

$ sudo update-initramfs -u
update-initramfs: Generating /boot/initrd.img-5.15.0-3-amd64
cryptsetup: WARNING: nvme0n1p3_crypt: ignoring unknown option 'fido2-device'

This is unfortunate. The simplest solution is to switch to dracut, a more modern initramfs generator, which among other things relies on systemd to activate encrypted volumes. This solves the issue of the unknown fido2-device.

Before installing dracut, I would highly recommend creating a copy of the existing initramfs in the boot partition in case something goes wrong.

$ sudo apt install dracut

Dracut includes systemd-cryptsetup by default. systemd-cryptsetup depends on libfido for unlocking devices using FIDO2 tokens. At least in Debian, systemd-cryptsetup dynamically loads libfido2.so (as opposed to being dynamically linked), which causes dracut not to have libfido2.so in the initramfs. This causes systemd-cryptsetup to issue the following error upon boot:

FIDO2 tokens not supported on this build. 

We fix it by manually adding libfido2.so to the initramfs. Of course, we also need to include libfido2’s dependencies as well. Dracut has a mechanism for automatically adding dependencies for executables, but it doesn’t work on libraries. As a workaround, instead of adding libfido2 directly, we will add an executable that depends on libfido2, which will add libfido2 and its dependencies to the initramfs. We will usefido2-token from the fido2-tools package for this trick.

$ sudo apt install fido2-tools
$ cat << EOF | sudo tee /etc/dracut.conf.d/11-fido2.conf
## Spaces in the quotes are critical.
# install_optional_items+=" /usr/lib/x86_64-linux-gnu/libfido2.so.* "

## Ugly workround because the line above doesn't fetch
## dependencies of libfido2.so
install_items+=" /usr/bin/fido2-token "

# Required detecting the fido2 key
install_items+=" /usr/lib/udev/rules.d/60-fido-id.rules /usr/lib/udev/fido_id "
EOF

Now, recreate the initramfs images:

$ sudo dracut -f

Last remarks

At this point, we are done. Reboot you’re machine and it will prompt you for your YubiKey and allow you to unlock your LUKS encrypted root patition with it. If you don’t have your YubiKey, it will give the following prompt:

Security token not present for unlocking volume root (nvme0n1p3_crypt), please plug it in.

After around 30 seconds, it would time out and display the following message:

Timed out waiting for security device, aborting security device based authentication attempt.

Afterwards, it would allow you to unlock the partition using a password (or a recovery key).

In case you run into any trouble, append rd.break=initqueue to the kernel command line, and dracut will enter a shell before attempting to mount the partitions. You can manually mount the drive using the following command:

# /usr/lib/systemd/systemd-cryptsetup attach root /dev/nvme0n1p3

Exit the emergency shell, and the system will continue its normal boot.

Extend laptop’s battery life using TLP

TLP helps optimize battery usage for laptops. TLP supports setting battery charging threshold, to allow keeping the battery partially charged, which prolongs its life. This can be done creating a new file under /etc/tlp.d/01-battery.conf:

START_CHARGE_THRESH_BAT0=80
STOP_CHARGE_THRESH_BAT0=85

Reload the TLP configuration for the new settings to take effect:

$ systemctl reload tlp.service

You can disable the battery threshold temporarily using:

$ sudo tlp fullcharge

Restoring the thresholds is done using

$ sudo tlp setcharge

Enable TRIM on external LUKS encrypted drive

If you use an encrypted external SSD, you should periodically trim it. The first step would be to make sure the external drive itself supports trimming. The next step would be to make sure the LUKS partition on the device supports trimming as well. By default, encrypted filesystems do not support passing discard requests due to some security concerns. For example, crypttab man page states:

WARNING: Assess the specific security risks carefully before enabling this option. For example, allowing discards on encrypted devices may lead to the leak of information about the ciphertext device (filesystem type, used space etc.) if the discarded blocks can be located easily on the device later.

For most users, the benefit of TRIM outweigh those security concerns. The easiest way to enable TRIM is to pass the discard option in /etc/crypttab. For example:

cdisk0 UUID=12345678-9abc-def012345-6789abcdef01 none luks,discard

The problem with the /etc/crypttab approach is that it requires you to pre-configure your external drives. A better approach would be to enable discards at the LUKS configuration, which would apply automatically whenever the drive is used. This can be done in LUKS version 2 headers.

# cryptsetup --allow-discards --persistent refresh luks-643dc0f7-c876-4e37-9207-5c053a75fc70

Where luks-643dc0f7-c876-4e37-9207-5c053a75fc70 is the name of the mapping for the encrypted drive. You can verify that allow_discards is now part of the flag by dumping the LUKS header.

# cryptsetup luksDump /dev/sda4 | grep Flags
Flags:       	allow-discard

Now, you should be able to use fstrim to trim your external SSD with LUKS encryption drive.

Enable TRIM/discard on external SSD

First, find out whether your device already supports TRIM commands.

$ lsblk --discard

Non-zero values in the DISC-GRAN and DISC-MAX indicate support. If it looks like your external SSD doesn’t support trimming, then maybe it supports UNMAP which is equivalent (UNMAP is just in the SCSI command set vs TRIM which is in the ATA command set). Assuming your external drive is /dev/sda

# apt install sg3-utils
# sg_vpd -a /dev/sda | grep -i unmap

If the last command has Unmap command supported (LBPU): 1 it means the drive supports the UNMAP command. If it’s supported, and discard wasn’t supported, it’s likely the kernel didn’t detect the UNMAP support. You can verify it by reading /sys/block/sda/device/scsi_disk/0\:0\:0\:0/provisioning_mode

$ cat /sys/block/sda/device/scsi_disk/0\:0\:0\:0/provisioning_mode 
full

full means no support. As we know our device supports unmap we can manually instruct the kernel about it.

# echo "unmap" >/sys/block/sda/device/scsi_disk/0\:0\:0\:0/provisioning_mode 

Now, lsblk --discard should report that the drive supports trimming, and you can use fstrim to trim it.

Making the change permanent

The changes above are ephemeral and will be reverted once you disconnect the drive. If you want to automatically apply those changes whenever your external drive is connected, we need to use udev rules.

Add the following rule to udev under /etc/udev/rules.d/90-usb-discard.rules

ACTION=="add|change", ATTRS{idVendor}=="0b05", ATTRS{idProduct}=="1932", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap"

Replace idVendor and idProduct above with the corresponding values for your device, as can be found in the output of lsusb.

Reload the udev rules using

# udevadm control --reload