Getting Started with Bootable Containers

Over the last decade, OCI containers have become a de facto way to deploy a complete functioning Linux user space as an application. A large set of practices and tooling have evolved around them. Bootable containers are a modern opinionated way of deploying, configuring and managing immutable image based Linux systems using those practices and tooling. You can find a mission statement on github.com/containers/bootable.

The bootc documentation summarizes bootable containers as "transactional, in-place operating system updates using OCI/Docker container images". In other words, updates to the operating system (OS) are shipped by using container images. That implies that the Linux kernel, the bootloader, drivers, etc. are all part of the container image which renders the container image "bootable".

Just like ordinary application containers, you can build bootable containers by using existing container technologies such as Containerfiles (i.e., Dockerfiles) and with existing tooling such as Podman, Docker or buildkit. You can further store the images on any container registry such as Quay.io, Docker Hub, the GitHub Container Registry or any internal container registry.

Bootable containers are a natural evolution of container technologies. For over a decade, containers have evolved into an industry standard of bundling, shipping, and deploying applications. Bootable containers built on top of these existing technologies and extend containers to include the entire operating system along with the Linux kernel to allow for a comprehensive container-native workflow and user experience.

Before going into any further details, you may wonder about the benefits of bootable containers and why should pay attention to this technology? Let’s go through some of the main benefits.

Bootc is at the core and center of bootable containers. It is a CLI tool that ships with a number of systemd services and timers to manage a bootable container. Among other things, bootc is responsible for downloading and queuing updates, and can be used by other higher-level tools to manage the system and inspect the system status. For that reason, bootc is an integral part of each bootable container image. For more details, please refer to the bootc documentation.

Bootc systems follow the concept of an immutable operating system. Apart from the following two exceptions, /etc and /var, all directories are mounted read-only once deployed on a physical or virtual machine. However, during a container build the entire file-system is writable.

The fact that most parts of the file system are mounted read-only is an important attribute of deployed bootable containers and something to consider carefully when preparing workloads and updates. For more information, see the Filesystem Layout page which elaborate in great detail on the exact behavior.

At the moment, there are three distributions shipping bootable containers:

The base images of Fedora and CentOS Stream are listed and continuously updated on the base images page. The RHEL bootc images can be found in the Red Hat Ecosystem Catalog; working with these requires a Red Hat account. You can get a no-cost subscription by joining the Red Hat Developer program in just a few clicks. You further need to login to the Red Hat Container Registry and register your machine with subscription-manager which is well explained in the release blog post. If you are using Podman Desktop, you might install the Red Hat Account Extension which automates most of the process.

As mentioned above, bootable containers can be built with existing tooling such as Containerfiles and Podman. That means you can use any existing bootc base image and customize it to your needs (e.g., install further packages, copy files from the host, run config scripts, etc.) in a container build as exemplified in the following Containerfile:

For more details on building derived bootc images, see the page in the docs. Note that bootc is still under development. On rare occasions you might encounter problems you can find in the upstream issue tracker (requires a GitLab account).

Updates for bootable containers happen in the form of pulling a new image and (re-)booting into it. But how do we install a fresh bootc system? While bootc supports installing a bootc container on top of an existing system, it is more common to convert a bootable container into a so-called disk image, such as ISO, raw or qcow2 to provision a new system.

You can convert a bootable container into a disk image with the bootc-image-builder which itself needs to be executed inside a container. bootc-image-builder is a feature-rich tool that further enables you to inject users, SSH keys, and define a partition layout. For more details, see the upstream documentation. Podman Desktop also ships with a bootc extension which allows you to build and convert bootc images in just a few clicks. For more information on the extension, please see the release blog post. Once converted, you can boot a disk image by using libvirt and qemu and other virtualization tools as described in in the docs.

If you are using Podman Desktop, you may install the Podman Desktop bootc extension to manage bootc images and automatically convert them to disk images.

As previously mentioned, bootable containers are OCI containers so they can be run via local container tools such as Podman or Docker. However, for fully-fledged tests it is not recommended to run a bootable container via, for instance, podman-run. One reason among others is that the filesystem is writable when being executed as an OCI container while most of the filesystem is mounted read-only on a deployed bootc system. That means the running container behaves differently than a deployed system. Yet, if you desire to run some quick tests it is recommended to run the container in detached mode. This way, systemd acts as the init process of the container and is able to run certain set up scripts used to initialize the system. You can then exec into a container as follows:

podman-bootc enables a more bootc-native experience. It is a CLI tool that allows you to easily run a local bootc image in a VM and get shell access to it. Under the hood, podman-bootc uses bootc-image-builder to create a disk image that is subsequently used to create and boot a VM. Before running podman-bootc, you must set up a Podman machine with a rootful connection. Then, the commands to run the Fedora bootc image via podman-bootc may look as follows:

podman-bootc is still under development but perfectly capable of supporting your development flow. For installation instructions and further details, please refer to the bootc upstream docs and section in the Fedora docs. If you are running on a Mac, please run brew install --head podman-bootc to build and install the latest version.

There are no default interactive users other than root in the base image. In the default full base images, OpenSSH is running, but there are no hardcoded credentials (passwords or SSH keys). That means that you cannot log into a booted VM without further work (e.g., injecting SSH keys for the root users). For that reason, podman-bootc automatically injects SSH keys such that you can easily get access to the VM. See the section on authentication in the docs for more information on this topic.

At this point, you know how to build a bootable container image and how to convert it into a disk image that you may want to install on your machine. Before installing Fedora/CentOS bootc, it’s recommended that you have created a customized derived container image, but this is not a hard requirement, as it is possible to enable basic system access via e.g. injecting SSH keys with kickstart or with bootc-install and the --root-ssh-authorized-keys flag. You can find a list of examples and detailed instructions here. You can further find documentation on how to provision public cloud instances on AWS or GCP and how to install on vSphere.

The figure below depicts the life cycle of a bootable container and the different steps required from building to deploying to updating bootc systems. Once a bootable container image has been built, you may convert it to a disk image. The disk image can then be used to install the content in the target environment (e.g., a public cloud instance). You might also want to push the container image to your target container registry.

To update existing systems, the process can be repeated: that is building a new image and pushing it to the registry. Once pushed, bootc on the deployed systems can pull down the new image from the registry and reboot into the new image.

There are several ways how an individual system can be updated:

Bootc further supports rollbacks via the bootc-rollback command. For more details, please refer to the auto-updates section which further elaborates on disconnected and bootloader updates.

The following hands-on demo shows the update process of a local VM:

At that point, we have learned the nature of and the concepts behind bootable containers. From building on top of available bootc base images to testing them locally with podman-bootc to understanding how to update already deployed systems.

The suggested next steps are to try out the technology and browse through the other pages of this documentation.

Bugs can be reported to:

You can find links to further community channels on the community page.