Container systemd-nspawn – an overview

Peter Boy, Jan Kuparinen Versio F34-F36 Last review: 2023-00-00

Status: Ready for review

The systemd-nspawn container runtime is part of the systemd system software. It has been offloaded into its own package, systemd-container, a while ago and must now installed separately. The toolset is powerful, yet lightweight and has low runtime overhead.

Miscellaneous (hopefully useful) information

The systemd management tools already used for the host system are applied to containers simply by specifying an additional parameter (-m <CONTAINER>), e.g. journalctl. Start and stop of containers is done using systemctl in the same way as all other system services. This makes the learning curve quite flat and maintenance quite simple.

The creation of a container filesystem or the provision of a corresponding image is treated as "out of scope" by systemd-nspawn. There are a number of options to generate it. By far the easiest and most efficient way is simply to use dnf in the container’s directory, and this is the recommended procedure. The final sections further down present some alternative options.

Storage considerations

The systemd-nspawn tools as machinctl first search for containers in /var/lib/machines/. The installation process automatically creates that directory. When setting up a container, create a subdirectory with the container name inside it and generate its directory tree therein. The directory is therefore directly container ROOT.

According to the default concept of Fedora Server disk partitioning, a separate logical volume should hold the machines directory. There are 2 valid alternatives to do that and one 'quick & dirty' solution:

The former is a good compromise between effort and reliability and therefore the recommended way.

(a) BTRFS Logical Volume

  • A logical volume of appropriate size with a BTRFS file system mounted ad /var/lib/machines holds all containers.

  • Setting up a new container creates a BTRFS subvolume instead of a simple subdirectory and thus provides a certain weak degree of separation and image-based backup per container.

  • Some systemd-nspawn tools can make use of specific BTRFS features and thus facilitate maintenance

  • A file system issue would affect all containers at the same time.

  • The system administrator must have confidence that BTRFS is sufficiently reliable for the intended purpose.

(b) Thin Provisioning XFS Volumes per Container

  • A thin provisioned logical volume of appropriate inital size with XFS file system mounted at /var/lib/machines/<CONTAINER> holds one container each.

  • Setting up a new container provides a strong data separation and image-based backup per container, but requires some administrative work.

  • A file system issue would affect just the specific container. Everything else would work fine.

  • With XFS, each container would use the currently most stable and best-performing generic file system available.

(c) XFS Logical Volume

  • A logical volume of appropriate size with a XFS file system mounted at /var/lib/machines holds all containers.

  • Setting up a new container creates a simple subdirectory and thus provides no separation of data and no image-based backup per container. Hence 'quick and dirty'

  • With XFS, containers would use the currently most stable and best-performing generic file system available.

The former is a good compromise between effort and reliability and therefore the recommended way.

Troubleshooting

SELinux blocks container startup as a system service

Issue

systemctl start <container> reports an AVC blockage

Solution

Just apply the usual SELinux correction procedure, preferably using Cockpit.

Root login fails

Issue

journalctl displays: pam_securetty(login:auth): access denied: tty 'pts/0' is not secure !

Solution

Delete /etc/securetty[7] and /usr/share/factory/etc/securetty on the container file system.

Using machinectl without root permission

Solution

requires polkit fix

Addendum

Containers of foreign distributions

Debian & Ubuntu

Fedora includes a customized version of the Debian / Ubuntu installer. This can be used to create the file system for corresponding containers. [source,]

[…]# dnf install debootstrap
[…]# debootstrap  {distro}  {targetdir}  {repo-url}

As an Ubuntu example

[…]# debootstrap  bionic /var/lib/machines/bionic http://archive.ubuntu.com/ubuntu

and a Debian example

[…]# debootstrap stable /var/lib/machines/deb http://dev.debian.org/debian

The installation process displays some error messages. These can be safely ignored.

Project mkosi

Program mkosi is an alternative to dnf to create a container directory tree.It is very comprehensive, has many more options than dnf and additionally offers customer image management features.

The project also provides pre-built images for various distributions, but these are not always maintained in a timely manner.

The project uses python and pip is an easy installation path.

[…]# yum install git
[…]# python3 -m pip install --user git+https://github.com/systemd/mkosi.git

For details see Lennart Poettering’s mkosi introduction, the mkosi man page, or the mkosi github repository

Want to help? Learn how to contribute to Fedora Docs ›