Tag: rsync

`rsync` and FAT File Systems

I’m using rsync to sync files from my computer to a FAT-formatted SD card. Using the --update flag with rsync makes it “skip files that are newer on the receiver.” It seems that it should work as follows: after the first sync, any subsequent syncs will only transfer those files that changed in the meantime. However, I noticed that transfer times are usually longer than expected, which led me to think that things are not working as they should.

As --update relies only on the modification date, obviously something is wrong with it. After ruling out several other possibilities, I guessed that the modification date of some files gets mangled. A quick check of FAT revealed that FAT can only save modification times with 2-second granularity. I’ve also used rsync’s --archive flag, which, among other things, attempts to preserve the modification times of the files it copies. But what about FAT’s lower granularity for modification time? That was apparently the culprit. Some files, when copied, got a modification time that was up to 1 second before the original modification time. Hence, every time I synced, from rsync’s perspective, the target was older than the source and therefore needed to be overwritten.

This can be demonstrated by the following session:
Continue reading `rsync` and FAT File Systems

Restricting SSH Access to rsync

Passphrase-less SSH keys allow one to automate remote tasks by not requiring user intervention to enter a passphrase to decrypt the key. While this is convenient, it poses a security risk, as the plain key can be used by anyone who gets hold of it to access the remote server. To this end, the developers of SSH made it possible to restrict, via .ssh/authorized_keys, the commands that can be executed by specific keys. This works great for simple commands, but because using rsync requires executing remote commands with different arguments on the remote end, depending on the invocation on the local machine, it gets quite complicated to properly restrict it via .ssh/authorized_keys.

Luckily, the developers of rsync foresaw this problem and wrote a script called rrsync (for restricted rsync) specifically to make it easier to restrict keys to be used only for rsync via .ssh/authorized_keys. If you have rsync installed, rrsync should have been distributed alongside it. On Debian/Ubuntu machines, it can be found under /usr/share/doc/rsync/scripts/rrsync.gz. If you cannot find it there, you can download the script directly from here. On the remote machine, copy the script, unpack it if needed, and make it executable:

user@remote:~$ gunzip /usr/share/doc/rsync/scripts/rrsync.gz -c > ~/bin/rrsync
user@remote:~$ chmod +x ~/bin/rrsync

On the local machine, create a new SSH key and leave the passphrase empty (this will allow you to automate the rsync via cron). Copy the public key to the remote server.

user@local:~$ ssh-keygen -f ~/.ssh/id_remote_backup -C "Automated remote backup"
user@local:~$ scp ~/.ssh/id_remote_backup.pub user@remote:~/

Once the public key is on the remote server, edit ~/.ssh/authorized_keys and append the public key.

user@remote:~$ vim ~/.ssh/authorized_keys

(Vim tip: Use :r! cat id_remote_backup.pub to directly insert the contents of id_remote_backup.pub into a new line). Now prepend the following to the newly added line:

command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding

The command="..." restricts access for that public key by executing the given command and disallowing others. All the other no-* stuff further restricts what can be done with that particular public key. As the SSH daemon will not start the default shell when accessing the server using this public key, the $PATH environment variable will be pretty empty (similar to cron), so you should specify the full path to the rrsync script. The two arguments to rrsync are -ro, which restricts modifying the directory (drop it if you want to upload stuff to the remote directory), and the path to the directory you want to enable remote access to (in my example ~/backups/).

The result should look something like:

command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAA...vp Automated remote backup

After saving the file, you should be able to rsync files from the remote server to the local machine, without being prompted for a password.

user@local:~$ rsync -e "ssh -i $HOME/.ssh/id_remote_backup" -av user@remote: etc2/

Two things need to be noted:

  1. You need to specify the passphrase-less key in the rsync command (the -e "ssh -i $HOME/.ssh/id_remote_backup" part).
  2. The remote directory is always relative to the directory given to rrsync in the ~/.ssh/authorized_keys file.