Old article

Hello, it turned out that this article is obsolete. I will rewrite it soon (soon for 2 August 2018), the security used in is not safe at all so the goal of this backup system isn’t achievable, thus it should not be used and I need another backup system.

One of the most important feature for me for dump was to keep track of the inodes numbers. Another solution is to save the list of the inodes numbers and their path in a file before doing a backup. This can be achieved with the following command.

doas ncheck -f "\I \P\n" /var

I planned to write an article about backup software as they are many of them in the OpenBSD ports tree, like borg, restic, duplicity, bacula, rsnapshot and others I did not mention.

Beginning of article

Today’s topic is “Encrypted backups” using only OpenBSD base tools. I am planning to write a bigger article later about backups but it’s a wide topic with a lot of software to cover and a lot of explanations about the differents uses cases, needs, issues an solutions. Here I will stick on explaining how to make reliable backups for an OpenBSD system (my laptop).

What we need is the dump command (see man 8 dump for its man page). It’s an utility to make a backup for a filesystem, it can only make a backup of one filesystem at a time. On my laptop I only backup /home partition so this solution is suitable for me while still being easy.

Dump can do incremental backups, it means that it will only save what changed since the last backup of lower level. If you do not understand this, please refer to the dump man page.

What is very interesting with dump is that it honors nodump flag which is an extended attribute of a FFS filesystem. One can use the command chflags nodump /home/solene/Downloads to tells dump not do save that folder (under some circumstances). By default, dump will not save thoses files, EXCEPT for a level 0 backup.

Important features of this backup solution:

• save files with attributes, permissions and flags
• can recreate a partition from a dump, restore files interactively, from a list or from its inode number (useful when you have files in lost+found)
• one dump = one file

My process is to make a huge dump of level 0 and keep it on a remote server, then, once a week I make a level 1 backup which will contain everything changed since the last dump of level 0, and everyday I do a level 2 backup of my files. The level 2 will contain latest files and the files changing a lot, which are often the most interesting. The level 1 backup is important because it will offload a lot of changes for the level 2.

Let me explain: let says my full backup is 60 GB, full of pictures, sources files, GUI applications data files etc… A level 1 backup will contain every new picture, new projects, new GUI files etc.. since the full backup, which will produce bigger and bigger dump over time, usually it is only 100 MB to 1GB. As I don’t add new pictures everyday or use new software everyday, the level 2 will take care of most littles changes to my data, like source code edited, little works on files etc… The level 2 backup is really small, I try to keep it under 50 MB so I can easily send it on my remote server everyday.

One could you more dump level, up to level 9, but keep in mind that those are incremental. In my case, if I need to restore all my partition, I will need to use level 0, 1 and 2 to get up to latest backup state. If you want to restore a file deleted a few days ago, you need to remember in which level its latest version is.

History note: dump was designed to be used with magnetic tapes.

Now, the interesting part: how to use it?

Commands to make a backup

The process is the following: dump | compression | openssl > file

To make a level 0 dump ignoring files having nodump flag:

dump -0 -h0 -a -u -f - /home

This will output the dump to stdout and ignore nodump files below level 0, which will ignore it whatever the current level is.

WARNING (POST PUBLICATION)

SOME PEOPLE REPORTED ME THAT USING THIS OPENSSL FEATURE IS NOT SAFE
AT ALL, PLEASE DO NOT USE THIS ON REMOTE BACKUPS YOU DO NOT TRUST
AS IT SHOULD BE EASY TO BRUTEFORCE THE PASSWORD. I NEED TO WRITE
A NEW WAY TO DO BACKUPS BUT I STILL NEED TO FIND A CORRECT WAY
TO MANAGE ENCRYPTION.

Now we will encrypt (with very low security) it with openssl to store securely the files on any media we could use (usb media, remote server, local server, another computer). We will use openssl command for this, with the password in the command line (this is not a problem for me as my computer is trustable).

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.enc

I choosed .enc extension file for encoded. You will need the password to read the file.

Now, we will see that using a compression tool before openssl can save a lot of space (depending of your data though). It is really easy to add compress in this pipe command.

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
gzip -f -c | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.gz.enc

In this case, gzip will save a lot of space if like me, most of your disk usage are mails and text files.

We can push the compression a little further as we want to reduce backup size for sending it to a remote server. We will use xz command with 2 threads to make it faster and disable checksum because anyway, openssl will allow to verify the integrity of the file.

LEVEL=0
dump -${LEVEL} -h0 -a -u -f - /home | \
xz -C none -T 2 -f -c | \
openssl enc -k "7H3_P@$$W0RD" -aes-256-cbc -salt -out dump-level${LEVEL}.xz.enc

How to restore from a dump

Now that we have a shiniy backup that we hope we will never have to use, it is important to understand to use it if needed.

The process is the following: cat file | openssl | decompression | restore

Before someone scream about the cat command, I know that I could use openssl file instead of but it feels more pleasant to read like this.

cat dump-level0.gz.enc | \
openssl enc -d -k "7H3_P@$$W0RD" -aes-256-cbc -salt - | \
xz -d -T2 -f -c - | \
restore -i -f -

One could write a short script like the following to give the file as a parameter and allow to choose restore’s parameters.

FILE=$1 ; shift
test -f "${FILE}" && cat "$1" | \
openssl enc -d -k "7H3_P@$$W0RD" -aes-256-cbc -salt - | \
xz -d -T2 -f -c - | \
restore $@ -f -

I have faced situations where restore should be called with differents flags, like -m to show inodes.

I hope you found this article interesting, I wanted to share a daily usage of simple tools which can give interesting features when combined together.

Yesterday I subscribed to a VPN service from the french association Grifon (Grifon website[FR] | gopher://grifon.fr) to get an IPv6 access to the world and play with IPv6. I will not talk about the VPN service in this article, it would be pointless.

So, I have an IPv6 prefix of 48 bytes which mean I can have a lot of addresses (I did some maths and found 65536² addresses but I am not sure about this).

Now, I would like my computer connected through the VPN to let others computers in my network to have IPv6 connectivity.

On OpenBSD, this only requires a few services, if you want to provide IPv6 to Windows devices on your network, you will need one more.

First, configure IPv6 on your lan

# ifconfig em0 inet6 autoconf

that’s all, you can add a new line “inet6 autoconf” to your file /etc/hostname.if to get it at boot.

Now, we have to allow IPv6 to be routed through the differents interfaces of the router.

# sysctl net.inet6.ip6.forwarding=1

This change can be made persistent across reboot by adding net.inet6.ip6.forwarding=1 to the file /etc/sysctl.conf.

Now we have to configure the daemon rtadvd to advertise the we are routing, devices on the network should be able to get an IPv6 address from its advertisement.

The minimal configuration of /etc/rtadvd.conf is the following:

em0:\
   :addr="2a00:5414:7311::":prefixlen#48:

In this configuration file, you have to type your IPv6 prefix in the addr field, and the prefix length in prefixlen. Others attributes could provide DNS servers to use for example.

Then enable the service at boot and start it:

# rcctl enable rtadvd
# rcctl set rtadvd flags em0
# rcctl start rtadvd

Tweaking resolv.conf

By default OpenBSD will ask for IPv4 when resolving a hostname (see syslog.conf(5) for more explanations). So, you will never have IPv6 traffic until you use a software which will request explicit IPv6 connection or that the hostname is only defined with a AAAA field.

# echo "family inet6 inet" >> /etc/resolv.conf.tail

The file resolv.conf.tail is appended at the end of resolv.conf when dhclient modifies the file resolv.conf.

Microsoft Windows

If you have Windows systems on your network, they won’t get addresses from rtadvd. You will need to deploy dhcpv6 daemon.

The configuration file for what we want to achieve here is pretty simple, it consists of telling what range we want to allow on DHCPv6 and a DNS server. Create the file /etc/dhcp6s.conf:

interface em0 {
    address-pool pool1 3600;
};
pool pool1 {
    range 2a00:5414:7311:1111::1000 to 2a00:5414:7311:1111::4000;
};
option domain-name-servers 2001:db8::35;

Note that I added “1111” into the range because it should not be on the same network than the router.

Now, you have to install and configure the service:

# pkg_add wide-dhcpv6
# echo SOME_RANDOM_CHARACTERS | openssl enc -base64 > /etc/dhcp6sctlkey
# chmod 400 /etc/dhcp6sctlkey
# echo "dhcp6s -c /etc/dhcp6s.conf  em0" >> /etc/rc.local

The openbsd package wide-dhcpv6 doesn’t provide a rc file to start/stop the service so it must be started from a command line, a way to do it is to type the command in /etc/rc.local which is run at boot. The openssl part is mandatory for dhcpv6 to start, it requires a base64 string as a secret key in the file /etc/dhcp6sctlkey.

This article will explain quickly how to bind a folder to access it from another path. It can be useful to give access to a specific folder from a chroot without moving or duplicating the data into the chroot.

Real world example: “I want to be able to access my 100GB folder /home/my_data/ from my httpd web server chrooted in /var/www/”.

The trick on OpenBSD is to use NFS on localhost. It’s pretty simple.

# rcctl enable portmap nfsd mountd
# echo "/home/my_data -network=127.0.0.0 -mask=255.255.255.0" > /etc/exports
# rcctl start portmap nfsd mountd

The order is really important. You can check that the folder is available through NFS with the following command:

$ showmount -e
Exports list on localhost:
/home/my_data               loopback

If you don’t have any line after “Exports list on localhost:”, you should kill mountd with pkill -9 mountd and start mountd again. I experienced it twice when starting all the daemons from the same commands but I’m not able to reproduce it. By the way, mountd only supports reload.

If you modify /etc/exports, you only need to reload mountd using rcctl reload mountd.

Once you have check that everything was alright, you can mount the exported folder on another folder with the command:

# mount localhost:/home/my_data /var/www/htdocs/my_data

You can add -ro parameter in the /etc/exports file on the export line if you want it to be read-only where you mount it.

Note: On FreeBSD/DragonflyBSD, you can use mount_nullfs /from /to, there is no need to setup a local NFS server. And on Linux you can use mount --bind /from /to and some others ways that I won’t cover here.

If you have enough memory on your system and that you can afford to use a few hundred megabytes to store temporary files, you may want to mount a mfs filesystem on /tmp. That will help saving your SSD drive, and if you use an old hard drive or a memory stick, that will reduce your disk load and improve performances. You may also want to mount a ramdisk on others mount points like ~/.cache/ or a database for some reason, but I will just explain how to achieve this for /tmp with is a very common use case.

First, you may have heard about tmpfs, but it has been disabled in OpenBSD years ago because it wasn’t stable enough and nobody fixed it. So, OpenBSD has a special filesystem named mfs, which is a FFS filesystem on a reserved memory space. When you mount a mfs filesystem, the size of the partition is reserved and can’t be used for anything else (tmpfs, as the same on Linux, doesn’t reserve the memory).

Add the following line in /etc/fstab (following fstab(5)):

swap /tmp mfs rw,nodev,nosuid,-s=300m 0 0

Frequently asked questions (with answers) on #openbsd IRC channel

Please read the official OpenBSD FAQ

I am writing this to answer questions asked too many times. If some answers get good enough, maybe we could try to merge it in the OpenBSD FAQ if the topic isn’t covered. If the topic is covered, then a link to the official FAQ should be used.

If you want to participate, you can fetch the page using gopher protocol and send me a diff:

$ printf '/~solene/article-openbsd-faq.txt\r\n' | nc dataswamp.org 70 > faq.md

What is the OpenBSD release process?

OpenBSD FAQ official information

The last two releases are called “-release” and are officially supported (patches for security issues are provided).

-stable version is the latest release with the base system patches applied, the -stable ports tree has some patches backported from -current, mainly to fix security issues. Official packages are not built for -stable. You have to build them yourself or install them using a third party service like M:Tier

What is -current?

It’s the development version with latest packages and latest code. You shouldn’t use it only to get latest package versions.

How do I install -current ?

OpenBSD FAQ about current

• download the latest snapshot install .iso or .fs file from your favorite mirror under /snapshots/ directory
• boot from it

How do I upgrade to -current

OpenBSD FAQ about current

• download the latest bsd.rd file from your favorite mirror
• verify its checksum and signature using signify and SHA256.sig file
• rename the old ramdisk kernel as /bsd.rd.old just in case
• copy the ramdisk kernel in its place - /bsd.rd
• reboot
• type “boot bsd.rd” at bootloader prompt
• choose “U” to upgrade
• reboot after the upgrade process
• pkg_add -u to update your packages

Repeat in order to upgrade to a newer -current snapshot

How do I update packages on my release version ?

Packages are frozen at the release and not updated.

When you fetch OpenBSD src or ports from CVS and that you want to save bandwidth during the process there is a little trick that change everything: compression

Just add -z9 to the parameter of your cvs command line and the remote server will send you compressed files, saving 10 times the bandwidth, or speeding up 10 times the transfer, or both (I’m in the case where I have differents users on my network and I’m limiting my incoming bandwidth so other people can have bandwidth too so it is important to reduce the packets transffered if possible).

The command line should looks like:

$ cvs -z9 -qd anoncvs@anoncvs.fr.openbsd.org:/cvs checkout -P src

Today I will explain how to do traffic limit with OpenBSD and PF. This is not hard at all if you want something easy, the man page pf.conf(5) in QUEUEING section is pretty good but it may disturbing when you don’t understand how it works. This is not something I master, I’m not sure of the behaviour in some cases but the following example works as I tested it ! :)

Use case

Internet is down at home, I want to use my phone as 4G router trough my OpenBSD laptop which will act as router. I don’t want the quota (some Gb) to be eaten in a few seconds, this connection allow to download up to 10 Mb/s so it can go quickly !

We will limit the total bandwidth to 1M (~ 110 kb/s) for people behind the NAT. It will be slow, but we will be sure that nothing behind the NAT like a program updating, cloud stuff synchronizing or videos in auto play won’t consume our quota.

Edit /etc/pf.conf accordigly to your network

internet="urndis0"
lan="em0"

# we define our available bandwidth
queue main on $lan bandwidth 100M

# we will let 1M but we will allow
# 3M during 200 ms when initiating connection to keep the web a bit interactive
queue limited parent main bandwidth 1M min 0K max 1M burst 3M for 200ms default

set skip on lo

# we do NAT here
match out on egress inet from !(egress:network) to any nat-to (egress:0)

block all
pass out quick inet

# we apply the queue here on EVERYTHING coming from the internet
pass in on $lan set queue limited

This ONLY defines queue for DOWNLOADING, you can only set the queue on the lan interface, this won’t work on egress (network interface having internet) because you can’t limit what go in your interface, it’s already there when you want to limit.

Per protocol ?

You can define queues per remote port by creating new queues and doing something like this:

pass in on $lan proto tcp port ssh set queue ssh
pass in on $lan proto tcp port www set queue web

Per host ?

As before, you can apply queues on IP host/range rather than protocols, or you can even mix both if you want.

Warning

The limit function changed in OpenBSD 5.5, everything you can read on the internet about ALTQ isn’t working anymore.

Today OpenBSD 6.1 has been released, I won’t copy & paste the change list but, in a few words, it gets better.

Link to the official announce

I already upgraded a few servers, with both methods. One with bsd.rd upgrade but that requires physical access to the server and the other method well explained in the upgrade guide which requires to untar the files and do move some files. I recommend using bsd.rd if possible.

Hello,

I have a pfsense appliance (Netgate 2440) with a usb console port, while it used to be a serial port, now devices seems to have a usb one. If you plug an usb wire from an openbsd box to it, you woull see this in your dmesg

uslcom0 at uhub0 port 5 configuration 1 interface 0 "Silicon Labs CP2104 USB to UART Bridge Controller" rev 2.00/1.00 addr 7
ucom0 at uslcom0 portno 0

To connect to it from OpenBSD, use the following command:

# cu -l /dev/cuaU0 -s 115200

And you’re done

Let’s encrypt is a free service which provides free SSL certificates. It is fully automated and there are a few tools to generate your certificates with it. In the following lines, I will just explain how to get a certificate in a few minutes. You can find more informations on Let’s Encrypt website.

To make it simple, the tool we will use will generate some keys on the computer, send a request to Let’s Encrypt service which will use http challenging (there are also dns and another one kind of challenging) to see if you really own the domain for which you want the certificate. If the challenge process is ok, you have the certificate.

Please, if you don’t understand the following commands, don’t type it.

While the following is right for OpenBSD, it may change slightly for others systems. Acme-client is part of the base system, you can read the man page acme-client(1).

Prepare your http server

For each certificate you will ask a certificate, you will be challenged for each domain on the port 80. A file must be available in a path under “/.well-known/acme-challenge/”.

You must have this in your httpd config file. If you use another web server, you need to adapt.

server "mydomain.com" {
    root "/empty"
    listen on * port 80
    location "/.well-known/acme-challenge/*" {
        root { "/acme/" , strip 2 }
    }
}

The “strip 2” part is IMPORTANT. (I’ve lost 45 minutes figuring out why root “/acme/” wasn’t working.)

Prepare the folders

As stated in acme-client man page and if you don’t need to change the path. You can do the following commands with root privileges :

# mkdir /var/www/acme
# mkdir -p /etc/ssl/acme/private /etc/acme
# chmod 0700 /etc/ssl/acme/private /etc/acme

Request the certificates

As root, in the acme-client sources folder, type the following the generate the certificates. The verbose flag is interesting and you will see if the challenging step work. If it doesn’t work, you should try manually to get a file like with the same path tried from Let’s encrypt, and try again the command when you succeed.

$ acme-client -vNn mydomain.com www.mydomain.com mail.mydomain.com

Use the certificates

Now, you can use your SSL certificates for your mail server, imap server, ftp server, http server…. There is a little drawback, if you generate certificates for a lot of domains, they are all written in the certificate. This implies that if someone visit one page, look at the certificate, this person will know every domain you have under SSL. I think that it’s possible to ask every certificate independently but you will have to play with acme-client flags and make some kind of scripts to automatize this.

Certificate file is located at /etc/ssl/acme/fullchain.pem and contains the full certification chain (as its name is explicit). And the private key is located at /etc/ssl/acme/private/privkey.pem.

Restart the service with the certificate.

Renew certificates

Certificates are valid for 3 months. Just type

./acme-client mydomain.com www.mydomain.com mail.mydomain.com

Restart your ssl services

EASY !

I am using the following lines in my /etc/sysctl.conf file, this boosted the performance on my multiples OpenBSD desktop:

kern.maxvnodes=768000
kern.maxfiles=32768
kern.maxclusters=256000
kern.seminfo.semmni=1024
kern.seminfo.semmns=4096
kern.shminfo.shmmax=805306368
kern.bufcachepercent=90

I want to talk about stumpwm, a window manager written in Common LISP. I think one must at least like emacs to like stumpwm. Stumpwm is a tiling window manager one which you create “panes” on the screen like windows on Emacs. A single pane takes 100% of the screen, then you can split it into 2 panes vertically or horizontally and resize it, and you can split again and again. There is no “automatic” tiling. By default, if you have ONE pane, you will only have ONE window displayed, this is a bit different that others tiling wm I had tried. Also, virtual desktops are named groups, nothing special here, you can create/delete groups and rename it. Finally, stumpwm is not minimalistic.

To install it, you need to get the sources of stumpwm, install a common lisp interpreter (sbcl, clisp, ecl etc…), install quicklisp (which is not in packages), install the quicklisp packages cl-ppcre and clx and then you can compile stumpwm, that will produce a huge binary which embedded a common lisp interpreter (that’s a way to share common lisp executables, the interpreter can create an executable from itself and include the files you want to execute). I would like to make a package for OpenBSD but packaging quicklisp and its packages seems too difficult for me at the moment.

Here is my config file in ~/.stumpwmrc.

Updated: 23th january 2018

(defun chomp(text) (subseq text 0 (- (length text) 1)))
(defmacro cmd(command) `(progn `(:eval (chomp (stumpwm:run-shell-command ,,command t)))))

(defun get-latence()
  (let ((now (get-universal-time)))
    (when (> (- now *latence-last-update* ) 30)
      (setf *latence-last-update* now)
      (when (probe-file "/tmp/latenceresult")
        (with-open-file (x "/tmp/latenceresult"
                           :direction :input)
          (setf *latence* (read-line x))))))
  *latence*)

(defvar *latence-last-update* (get-universal-time))
(defvar *latence* "nil")


(set-module-dir "~/dev/stumpwm-contrib/")
(stumpwm:run-shell-command "setxkbmap fr")
(stumpwm:run-shell-command "feh --bg-fill red_damask-wallpaper-1920x1080.jpg")

(defvar color1 "#886666")
(defvar color2 "#222222")

(setf
 stumpwm:*mode-line-background-color* color2 
 stumpwm:*mode-line-foreground-color* color1
 stumpwm:*mode-line-border-color* "#555555"
 stumpwm:*screen-mode-line-format* (list "%g | %v ^>^7 %B | " '(:eval (get-latence)) "ms %d    ")
 stumpwm:*mode-line-border-width* 1
 stumpwm:*mode-line-pad-x* 6
 stumpwm:*mode-line-pad-y* 1
 stumpwm:*mode-line-timeout* 5
 stumpwm:*mouse-focus-policy* :click
 ;;stumpwm:*group-format* "%n·%t
 stumpwm:*group-format* "%n"
 stumpwm:*time-modeline-string* "%H:%M"
 stumpwm:*window-format* "^b^(:fg \"#7799AA\")<%25t>"
 stumpwm:*window-border-style* :tight
 stumpwm:*normal-border-width* 1
 )


(stumpwm:set-focus-color "#7799CC")
(stumpwm:grename "Alpha")
(stumpwm:gnewbg "Beta")
(stumpwm:gnewbg "Tau")
(stumpwm:gnewbg "Pi")
(stumpwm:gnewbg "Zeta")
(stumpwm:gnewbg "Teta")
(stumpwm:gnewbg "Phi")
(stumpwm:gnewbg "Rho")

(stumpwm:toggle-mode-line (stumpwm:current-screen) (stumpwm:current-head))

(set-prefix-key (kbd "M-a"))

(define-key *root-map* (kbd "c")            "exec urxvtc")
(define-key *root-map* (kbd "RET")          "move-window down")
(define-key *root-map* (kbd "z")            "fullscreen")

(define-key *top-map* (kbd "M-&")           "gselect 1")
(define-key *top-map* (kbd "M-eacute")      "gselect 2")
(define-key *top-map* (kbd "M-\"")          "gselect 3")
(define-key *top-map* (kbd "M-quoteright")  "gselect 4")
(define-key *top-map* (kbd "M-(")           "gselect 5")
(define-key *top-map* (kbd "M--")           "gselect 6")
(define-key *top-map* (kbd "M-egrave")      "gselect 7")
(define-key *top-map* (kbd "M-underscore")  "gselect 8")

(define-key *top-map* (kbd "s-l")           "exec slock")
(define-key *top-map* (kbd "s-t")           "exec urxvtc")
(define-key *top-map* (kbd "M-S-RET")       "exec urxvtc")
(define-key *top-map* (kbd "M-C")           "exec urxvtc")

(define-key *top-map* (kbd "s-s")           "exec /home/solene/dev/screen_up.sh")

(define-key *top-map* (kbd "s-Left")        "gprev")
(define-key *top-map* (kbd "s-Right")       "gnext")

(define-key *top-map* (kbd "M-ISO_Left_Tab")"other")
(define-key *top-map* (kbd "M-TAB")         "fnext")
(define-key *top-map* (kbd "M-twosuperior") "next-in-frame")

(load-module "battery-portable")
(load-module "stumptray")

I use a function to get latency from a script that is started every 20 seconds to display the network latency or nil if I don’t have internet access.

I use rxvt-unicode daemon (urxvtd) as a terminal emulator, so the terminal command is urxvtc (for client), it’s lighter and faster to load.

I also use a weird “alt+tab” combination:

• Alt+tab switch between panes
• Alt+² (the key above tab) circles windows in the current pane
• Alt+Shift+Tab switch to the previous windows selected

StumpWM website