About me: My name is Solène Rapenne, pronouns she/her. I like learning and sharing knowledge. Hobbies: '(BSD OpenBSD Qubes OS Lisp cmdline gaming security QubesOS internet-stuff). I love percent and lambda characters. Qubes OS core team member, former OpenBSD developer solene@. No AI is involved in this blog.

Contact me: solene at dataswamp dot org or @solene@bsd.network (mastodon).

I'm a freelance OpenBSD, FreeBSD, Linux and Qubes OS consultant, this includes DevOps, DevSecOps, technical writing or documentation work. If you enjoy this blog, you can sponsor my open source work financially so I can write this blog and contribute to Free Software as my daily job.

How to run a NixOS VM as an OpenBSD guest

Written by Solène, on 08 May 2021.
Tags: #openbsd #nixos

Comments on Fediverse/Mastodon

1. Introduction §

This guide is to help people installing the NixOS Linux distribution as a virtual machine guest hosted on OpenBSD VMM hypervisor.

2. Preparation §

Some operations are required on the host but specifics instructions will be needed on the guest as well.

2.1. Create the disk §

We will create a qcow2 disk, this format allows not using all the reserved space upon creation, size will grow as the virtual disk will be filled with data.

vmctl create -s 20G nixos.qcow2

2.2. Configure vmd §

We have to configure the hypervisor to run the VM. I've chose to define a new MAC address for the VM interface to avoid collision with the host MAC.

vm "nixos" {
       memory 2G
       disk "/home/virt/nixos.qcow2"
       cdrom "/home/virt/latest-nixos-minimal-x86_64-linux.iso"
       interface { lladdr "aa:bb:cc:dd:ee:ff"  switch "uplink" }
       owner solene
       disable
}

switch "uplink" {
	interface bridge0
}

vm.conf man page

2.3. Configure network §

We need to create a bridge in which I will add my computer network interface "em0" to it. Virtual machines will be attached to this bridge and will be seen from the network.

echo "add em0" > /etc/hostname.bridge0
sh /etc/netstart bridge0

2.4. Start vmd §

We want to enable and then start vmd to use the virtual machine.

rcctl enable vmd
rcctl start vmd

2.5. NixOS and serial console §

When you are ready to start the VM, type "vmctl start -c nixos", you will get automatically attached to the serial console, be sure to read the whole chapter because you will have a time frame of approximately 10 seconds before it boots automatically (if you don't type anything).

If you see the grub display with letters displayed more than once, this is perfectly fine. We have to tell the kernel to enable the console output and the desired speed.

On the first grub choice, press "tab" and append this text to the command line: "console=ttyS0,115200" (without the quotes). Press Enter to validate and boot, you should see the boot sequence.

For me it took a long time on starting sshd, keep waiting, that will continue after less than a few minutes.

2.6. Installation §

There is an excellent installation guide for NixOS in their official documentation.

Official installation guide

I had issues with DHCP so I've set the network manually, my network is in 192.168.1.0/24 and my router 192.168.1.254 is offering DNS too.

systemctl stop NetworkManager
ifconfig enp0s2 192.168.1.151/24 up
route add -net default gw 192.168.1.254
echo "nameserver 192.168.1.254" >> /etc/resolv.conf

The installation process can be summarized with theses instructions:

sudo -i
parted /dev/vda -- mklabel msdos
parted /dev/vda -- mkpart primary 1MiB -1GiB # use every space for root except 1 GB for swap
parted /dev/vda -- mkpart primary linux-swap -1GiB 100%
mkfs.xfs -L nixos /dev/vda1
mkswap -L swap /dev/vda2
mount /dev/disk/by-label/nixos /mnt
swapon /dev/vda2
nixos-generate-config --root /mnt
nano /mnt/etc/nixos/configuration.nix
nixos-install
shutdown now

Here is my configuration.nix file on my VM guest, it's the most basic I could want and I stripped all the comments from the base example generated before install.

{ config, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  boot.loader.grub.enable = true;
  boot.loader.grub.version = 2;
  boot.loader.grub.extraConfig = ''
    serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
    terminal_input --append serial
    terminal_output --append serial
  '';

  networking.hostName = "my-little-vm";
  networking.useDHCP = false;

  # COMMENT THIS LINE IF YOU DON'T WANT DHCP
  # networking.interfaces.enp0s2.useDHCP = true;


  # BEGIN ADDITION
  # all of these variables were added or uncommented
  boot.loader.grub.device = "/dev/vda";

  # required for serial console to work!
  boot.kernelParams = [
    "console=ttyS0,115200n8"
  ];

  systemd.services."serial-getty@ttyS0" = {
    enable = true;
    wantedBy = [ "getty.target" ]; # to start at boot
    serviceConfig.Restart = "always"; # restart when session is closed
  };

  # use what you want
  time.timeZone = "Europe/Paris";

  # BEGIN NETWORK
  # define network here
  networking.interfaces.enp0s2.ipv4.addresses = [ {
        address = "192.168.1.151";
        prefixLength = 24;
  } ];
  networking.defaultGateway = "192.168.1.254";
  networking.nameservers = [ "192.168.1.254" ];
  # END NETWORK

  # enable SSH and allow X11 Forwarding to work
  services.openssh.enable = true;
  services.openssh.forwardX11 = true;

  # Declare a user that can use sudo
  users.users.solene = {
    isNormalUser = true;
    extraGroups = [ "wheel" ];
  };

  # declare the list of packages you want installed globally
  environment.systemPackages = with pkgs; [
     wget vim
  ];

  # firewall configuration, only allow inbound TCP 22
  networking.firewall.allowedTCPPorts = [ 22 ];
  networking.firewall.enable = true;
  # END ADDITION

  # DONT TOUCH THIS EVER EVEN WHEN UPGRADING
  system.stateVersion = "20.09"; # Did you read the comment?

}

Edit /etc/vm.conf to comment the cdrom line and reload vmd service. If you want the virtual machine to automatically start with vmd, you can remove the "disable" keyword.

Once your virtual machine is started again with "vmctl start nixos", you should be able to connect to ssh to it. If you forgot to add users, you will have to access the VM console with "vmctl console", log as root, modify the configuration file, type "nixos-rebuild switch" to apply changes, and then "passwd user" to define the user password. You can set a public key when declaring a user if you prefer (I recommend).

3. Install packages §

There are three ways to install packages on NixOS: globally, per-user or for a single run.

- globally: edit /etc/nixos/configuration.nix and add your packages names to the variable "environment.systemPackages" and then rebuild the system

- per-user: type "nix-env -i nixos.firefox" to install Firefox for that user

- for single run: type "nix-shell -p firefox" to create a shell with Firefox available in it

Note that the single run doesn't mean the package will disappear, it's most likely... not "hooked" into your PATH so you can't use it. This is mostly useful when you make development and you need specific libraries to build a project and you don't always want them available for your user.

4. Conclusion §

While I never used a Linux system as a guest in OpenBSD it may be useful to run Linux specific software occasionally. With X forwarding, you can run Linux GUI programs that you couldn't run on OpenBSD, even if it's not really smooth it may be enough for some situations.

I chose NixOS because it's a Linux distribution I like and it's quite easy to use in the regards it has only one configuration file to manage the whole system.