Old article

Hello, it turned out that this article is obsolete. I will rewrite it soon (soon for 2 August 2018), the security used in is not safe at all so the goal of this backup system isn’t achievable, thus it should not be used and I need another backup system.

One of the most important feature for me for dump was to keep track of the inodes numbers. Another solution is to save the list of the inodes numbers and their path in a file before doing a backup. This can be achieved with the following command.

doas ncheck -f "\I \P\n" /var

I planned to write an article about backup software as they are many of them in the OpenBSD ports tree, like borg, restic, duplicity, bacula, rsnapshot and others I did not mention.

Beginning of article

Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).

What we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.

Dump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.

What is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.

Important features of this backup solution:

• save files with attributes, permissions and flags
• can recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found)
• one dump = one file

My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.

Let me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc.. since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.

One could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.

History note: dump was designed to be used with magnetic tapes.

Now, the interesting part: how to use it?

Commands to make a backup

The process is the following: dump | compression | openssl > file

To make a level 0 dump ignoring files having nodump flag:

dump -0 -h0 -a -u -f - /home

This will output the dump to stdout and ignore nodump files below level 0, which will ignore it whatever the current level is.

WARNING (POST PUBLICATION)

SOME PEOPLE REPORTED ME THAT USING THIS OPENSSL FEATURE IS NOT SAFE
AT ALL, PLEASE DO NOT USE THIS ON REMOTE BACKUPS YOU DO NOT TRUST
AS IT SHOULD BE EASY TO BRUTEFORCE THE PASSWORD. I NEED TO WRITE
A NEW WAY TO DO BACKUPS BUT I STILL NEED TO FIND A CORRECT WAY
TO MANAGE ENCRYPTION.

Now we will encrypt (with very low security) it with openssl to store securely the files on any media we could use (usb media, remote server, local server, another computer). We will use openssl command for this, with the password in the command line (this is not a problem for me as my computer is trustable).

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.enc

I choosed .enc extension file for encoded. You will need the password to read the file.

Now, we will see that using a compression tool before openssl can save a lot of space (depending of your data though). It is really easy to add compress in this pipe command.

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
gzip -f -c | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.gz.enc

In this case, gzip will save a lot of space if like me, most of your disk usage are mails and text files.

We can push the compression a little further as we want to reduce backup size for sending it to a remote server. We will use xz command with 2 threads to make it faster and disable checksum because anyway, openssl will allow to verify the integrity of the file.

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
xz -C none -T 2 -f -c | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.xz.enc

How to restore from a dump

Now that we have a shiniy backup that we hope we will never have to use, it is important to understand to use it if needed.

The process is the following: cat file | openssl | decompression | restore

Before someone scream about the cat command, I know that I could use openssl file instead of but it feels more pleasant to read like this.

cat dump-level0.gz.enc | \
openssl enc -d -k "7H3_P@$$W0RD" -aes-256-cbc -salt - | \
xz -d -T2 -f -c - | \
restore -i -f -

One could write a short script like the following to give the file as a parameter and allow to choose restore’s parameters.

FILE=$1 ; shift
test -f "${FILE}" && cat "$1" | \
openssl enc -d -k "7H3_P@$$W0RD" -aes-256-cbc -salt - | \
xz -d -T2 -f -c - | \
restore $@ -f -

I have faced situations where restore should be called with differents flags, like -m to show inodes.

I hope you found this article interesting, I wanted to share a daily usage of simple tools which can give interesting features when combined together.