Setting xattrs: fsetxattr(security.selinux): Operation not supported

[brianwcook]

I am looking to assess the ability of our new container build system to build ostree native containers. To get a baseline, I am just trying to build a simple image on a laptop for now. I cloned https://pagure.io/workstation-ostree-config.git and I am trying to build fedora-base with the following command:

rpm-ostree compose image
--initialize-mode=if-not-exists
--format=registry
../workstation-ostree-config/fedora-base.yaml
quay.io/bcook/ostree:fedora-base-test

I tried it two ways: in toolbox as root, and in a privileged container as root. Both ways I received the following error:

Running post scripts... done
error: While applying overrides for pkg fuse3: Copyup usr/bin/fusermount3: Setting xattrs: fsetxattr(security.selinux): Operation not supported
error: compose tree failed: ExitStatus(unix_wait_status(256))

Couldn't find much info that matched this. Hoping someone can give me a pointer. Thanks!

[cgwalters]

I tried it two ways: in toolbox as root

OK yes, basically you need to make /var/tmp a real filesystem and not an overlayfs. Does it work to do sudo mount --bind /run/host/var/tmp/ /var/tmp/?

Unfortunately today we require support for setting the security.selinux xattr. The easiest way to handle this is to have the cache filesystem be a real filesystem.

We should definitely make this requirement much more obvious; e.g. verify that if selinux is enabled we can set xattrs in the cache root.

[brianwcook]

I'm trying to build inside a container. Is this something you've managed to do? It seems like this limitation prevents it.

[cgwalters]

Yes. Did you see my comment above?

Does it work to do sudo mount --bind /run/host/var/tmp/ /var/tmp/?

(In your toolbox environment)

[brianwcook]

ok, I was able to get a build to work in podman. Neither overlayfs nor tmpfs supports xattrs. What worked was on regular fedora38:

mkdir ~/tmp podman run --privileged --mount type=bind,source=/home/bcook/tmp,target=/var/tmp,relabel=shared -it a315f4cc76e6 /bin/bash

Now, next hurdle: any chance we can make this work without --privileged? If I remove it I immediately get

bwrap: Creating new namespace failed: Operation not permitted error: bwrap test failed, see <https://github.com/coreos/rpm-ostree/pull/429>: bwrap(true): Child process killed by signal 1 error: compose tree failed: ExitStatus(unix_wait_status(256))

[cgwalters]

Ultimately rpm-ostree uses container technology itself today. So there's two options:

• Full privileges (not ideal obviously)
• In Kubernetes/OCP, we can use user namespaces - I haven't yet tried this but would like to
• Run in virtualization (which we've historically often been doing because we wanted to make disk images too); this is how https://github.com/coreos/coreos-assembler is oriented

[brianwcook]

if we need privileged container support we will farm those jobs out to ephemeral VM's. At that point we could run either rpm-ostree or core-os assembler. Is there a reason to support both? Should we focus on coreos assembler?

[cgwalters]

We're really super close to being runnable with user namespaces enabled now last I looked. Would probably just need a bit of work. But yes.

Should we focus on coreos assembler?

No; for future work we're going to focus on what is today rpm-ostree compose image, not coreos-assembler which also makes disk images, which we don't want to involve in this yet.