Random Musings

O for a muse of fire, that would ascend the brightest heaven of invention!

A Brief Introduction to OCI Containers on FreeBSD

Wednesday, 4 Dec 2024 Tags: containersfreebsdocipodman

OCI Containers on FreeBSD

These Work-in-progress notes will be updated over time, and merged into the FreeBSD HandBook once they are complete. For the moment, have fun, and send feedback to dch@FreeBSD.org

14.2-RELEASE now includes OCI-compatible images, and the Podman toolkit on FreeBSD is ready to use them, on both amd64 and arm64 systems.

In the aarch64 and amd64 download directories, you’ll see 3 official OCI-format images. The naming may be a little confusing at first, but should make sense once you start using them.

Each image comprises a subset of a standard FreeBSD base.txz release tarball, for various use cases.

CHECKSUM.SHA256
CHECKSUM.SHA512
FreeBSD-14.2-RELEASE-amd64-container-image-dynamic.txz
FreeBSD-14.2-RELEASE-amd64-container-image-minimal.txz
FreeBSD-14.2-RELEASE-amd64-container-image-static.txz

static

The static image is the bare minimum functional system, if you have a statically compiled binary, this should be able to run using only the static image.

  • only the bare bones
  • TLS certs
  • minimal password file
  • minimal termcap
  • tzdata file
  • just enough to run a static binary

Note that it has, by design, a lean footprint, to make a security compromise of the container less useful to the attacker. There’s no UNIX shell, no command-line tools, nor a package manager.

dynamic

The dynamic image uses the static image as a parent layer, and adds enough bits to be able to use shared libraries, and libc.

It should be able to run most binaries and tools, but still doesn’t have a shell, nor a package manager.

minimal

Again, minimal builds on the preceding dynamic layer, and finally adds the minimum that a user would expect - a UNIX shell, and package.

scratch and builder

There are two other common images that are worth mentioning - scratch as the /dev/null of images, is completely empty. You typically use this to construct very specific containers for your own systems.

A builder container is typically used as an intermediate step that enables you to create, as an output, the final container you wish to run in production. The builder might have compilers, and other build tools necessary for your organisation, even if the final image, does not.

A Brief Introduction to Podman

The podman suits comprises 3 main tools and a bunch of supporting ones:

  • podman is a daemonless container engine using docker-compatible commands
  • buildah generates OCI formatted container images
  • skopeo works with remote OCI image registries

Mad props to SamuelKarp@ and dfr@ who made all of this work, from building https://github.com/samuelkarp/runj the first container-friendly port for FreeBSD, to porting the entire podman suite and getting necessary kernel changes to support it. Mad Mad Props!

Installing Podman

  • remember to run all this in screen/tmux/mosh to avoid network loss while restarting your firewall
  • integrate changes from /usr/local/etc/containers/pf.conf.sample into your /etc/pf.conf setting egress macros appropriately
  • amend sysctl.conf, and /etc/fstab as appropriate 
# zfs create -o mountpoint=/var/db/containers zroot/containers
# zfs snapshot zroot/containers@empty
# sysctl net.pf.filter_local=1
# mount -t fdescfs fdesc /dev/fd
# pkg install -r FreeBSD -y podman-suite emulators/qemu-user-static
# service pf restart

If you don’t have zfs, switch to ufs:

# sed -I .bak -e 's/driver = "zfs"/driver = "vfs"/' \
    /usr/local/etc/containers/storage.conf
  • review and amend the template config files. None of these are initially required though: 
# pkg list buildah podman conmon  ocijail containers-common \
    containernetworking-plugins \
    |grep /etc/
/usr/local/etc/containers/containers.conf.sample
/usr/local/etc/containers/policy.json.sample
/usr/local/etc/containers/registries.conf.sample
/usr/local/etc/containers/storage.conf.sample
/usr/local/etc/containers/pf.conf.sample

A Quick Terminal Tour

Contrary to podman/docker on Linux, everything needs to run as root. For the moment.

Now that we have the necessary tools and firewall rules in place, let’s take it out for a spin, fetching the officially published images from 14.2-RELEASE, either aarch64 or amd64 as appropriate.

Fetch the Official Images

Note how the first image already contains all the blobs needed for the smaller images - the archives share common blobs within the store. You can fetch the exported images and compare the checksums against the 14.2-RELEASE announcement to be certain, or you can verify the image checksums against the ones show below, the end result is the same.

The blob truncated checksums below will be the same on all images in any registry. While these aren’t yet published as part of the release process, they should be for future releases, making this step even easier to verify.

# export OCIBASE=https://download.freebsd.org/releases/OCI-IMAGES/14.2-RELEASE/amd64/Latest
# podman load -i=$OCIBASE/FreeBSD-14.2-RELEASE-amd64-container-image-minimal.txz
Getting image source signatures
Copying blob 3d72902df126 done   |
Copying blob 9154eccec9cd done   |
Copying blob 6c3cf2b1cc8a done   |
Copying config c5f3e77557 done   |
Writing manifest to image destination
Loaded image: localhost/freebsd14-minimal:14.2-RELEASE-amd64

# podman load -i=$OCIBASE/FreeBSD-14.2-RELEASE-amd64-container-image-dynamic.txz
Getting image source signatures
Copying blob 3d72902df126 skipped: already exists
Copying blob 9154eccec9cd skipped: already exists
Copying config ebf7538b22 done   |
Writing manifest to image destination
Loaded image: localhost/freebsd14-dynamic:14.2-RELEASE-amd64

# podman load -i=$OCIBASE/FreeBSD-14.2-RELEASE-amd64-container-image-static.txz
Getting image source signatures
Copying blob 3d72902df126 skipped: already exists
Copying config 7876fe59db done   |
Writing manifest to image destination
Loaded image: localhost/freebsd14-static:14.2-RELEASE-amd64

List the Images

  • the image IDs are reproducible
  • and available in buildah and podman 
# podman images
REPOSITORY                   TAG                 IMAGE ID      CREATED     SIZE
localhost/freebsd14-minimal  14.2-RELEASE-amd64  c5f3e77557a9  4 days ago  35.1 MB
localhost/freebsd14-dynamic  14.2-RELEASE-amd64  ebf7538b22f4  4 days ago  15.9 MB
localhost/freebsd14-static   14.2-RELEASE-amd64  7876fe59dbb3  4 days ago  5.45 MB
# buildah images
REPOSITORY                    TAG                  IMAGE ID       CREATED      SIZE
localhost/freebsd14-minimal   14.2-RELEASE-amd64   c5f3e77557a9   4 days ago   35.1 MB
localhost/freebsd14-dynamic   14.2-RELEASE-amd64   ebf7538b22f4   4 days ago   15.9 MB
localhost/freebsd14-static    14.2-RELEASE-amd64   7876fe59dbb3   4 days ago   5.45 MB

List the Layers

Note how showing the layers for the upper image, called minimal, shows how it is built on top of the lower layers - each layer shares the same components.

# podman image tree localhost/freebsd14-minimal:14.2-RELEASE-amd64
Image ID: c5f3e77557a9
Tags:     [localhost/freebsd14-minimal:14.2-RELEASE-amd64]
Size:     35.07MB
Image Layers
├── ID: cd53fb07fb66 Size: 5.449MB Top Layer of: [localhost/freebsd14-static:14.2-RELEASE-amd64]
├── ID: a01d37f7777b Size:  10.4MB Top Layer of: [localhost/freebsd14-dynamic:14.2-RELEASE-amd64]
└── ID: 36b0c80ca1f7 Size: 19.21MB Top Layer of: [localhost/freebsd14-minimal:14.2-RELEASE-amd64]

Inspect the Checksums

These sha256 digests can be compared across hosts, to ensure you are getting the expected images.

# podman inspect localhost/freebsd14-minimal:14.2-RELEASE-amd64
[
     {
          "Id": "c5f3e77557a916c124bfb74bdd982efe0a5749aaba9de0d282703d4634b28839",
          "Digest": "sha256:b98b8f5438a07e000a156ecb216529d8dfa191448ac3dc1f6add9848b4461d05",
          "RepoTags": [
               "localhost/freebsd14-minimal:14.2-RELEASE-amd64"
          ],
          "RepoDigests": [
               "localhost/freebsd14-minimal@sha256:b98b8f5438a07e000a156ecb216529d8dfa191448ac3dc1f6add9848b4461d05"
          ],
          "Parent": "ebf7538b22f43206d308fecc12bfebc72beb6d1f0e89ce9d95ed1979583010ee",
          "Comment": "",
          "Created": "2024-11-29T10:34:57.587490327Z",
          "Config": {
               "Labels": {
                    "io.buildah.version": "1.36.0"
               }
          },
          "Version": "",
          "Author": "",
          "Architecture": "amd64",
          "Os": "freebsd",
          "Size": 35067392,
          "VirtualSize": 35067392,
          "GraphDriver": {
               "Name": "zfs",
               "Data": {
                    "Dataset": "zroot/containers/36b0c80ca1f71afdd5813c5fbd5b55fe686ec6361df1ff36286f2176be67a3fe",
                    "Mountpoint": "/var/db/containers/storage/zfs/graph/36b0c80ca1f71afdd5813c5fbd5b55fe686ec6361df1ff36286f2176be67a3fe"
               }
          },
          "RootFS": {
               "Type": "layers",
               "Layers": [
                    "sha256:cd53fb07fb66f2c92709cbc407af686fa53276397762dfb27c040a2f64508000",
                    "sha256:f709c8f6cb8695415033da5819b83262c51cd9512bddb64e0681c48a61b8723f",
                    "sha256:357c210f8575c70360e88cfcf3d0c241f7e3fa21669ec55bb935dce361a90585"
               ]
          },
          "Labels": {
               "io.buildah.version": "1.36.0"
          },
          "Annotations": {
               "org.opencontainers.image.base.digest": "sha256:7c1c45d99ffe8344021227ad651f675ac22eeb39b96c8c6abfe2d0612f4c2ccc",
               "org.opencontainers.image.base.name": "localhost/freebsd14-dynamic:latest"
          },
          "ManifestType": "application/vnd.oci.image.manifest.v1+json",
          "User": "",
          "History": [
               {
                    "created": "2024-11-29T09:55:25.405122405Z",
                    "created_by": "/bin/sh"
               },
               {
                    "created": "2024-11-29T10:34:08.010623771Z",
                    "created_by": "/bin/sh",
                    "comment": "FROM localhost/freebsd14-static:latest"
               },
               {
                    "created": "2024-11-29T10:34:58.10725353Z",
                    "created_by": "/bin/sh",
                    "comment": "FROM localhost/freebsd14-dynamic:latest"
               }
          ],
          "NamesHistory": [
               "localhost/freebsd14-minimal:14.2-RELEASE-amd64"
          ]
     }
]

Run A Container

# podman run -it localhost/freebsd14-minimal:14.2-RELEASE-amd64 /bin/sh
# uname -a
FreeBSD 3fad4ca90729 15.0-CURRENT FreeBSD 15.0-CURRENT main-n273934-3455b824a1e8 GENERIC-NODEBUG amd64
# freebsd-version -kru
freebsd-version: unable to locate kernel
# freebsd-version -ru
15.0-CURRENT
14.2-RELEASE
# exit