Drop requirement on nss-altfiles, use systemd sysusers

[cgwalters]

Currently, all %posts are run on the server side. This means we need nss-altfiles to help OSTree's 3 way merge of /etc.

The systemd people added systemd-sysusers, with the intention that the updates happen on boot.

A new hybrid option is running systemd-sysusers in the new root just before reboot. This would avoid system mutation on boot with all the problems that entails (e.g. booting read-only should work, etc.)

[cgwalters]

See also #40

[cgwalters]

Here's an idea too; we don't necessarily need to force all RPMs to use sysusers. We could synthesize sysusers entries from them, as long as we know systemd-sysusers is in the target OS.

And in fact, we may need to synthesize anyways, for the case where the tree content contains uid-specific data (e.g. /usr/lib64/dbus-1/dbus-daemon-launch-helper, /etc/polkit-1/rules.d). For those cases we have to pick a uid at compose time, then tell sysusers to match that uid.

[xnox]

How about fixing shadow-utils to be nss-altfiles aware? That is read altfiles groups, when doing append operations to them?

[cgwalters]

I wrote a patch for that: http://lists.alioth.debian.org/pipermail/pkg-shadow-devel/2014-April/010149.html

However at the time, it was painful to depend on the shadow-utils patch. It still isn't applied upstream. I stopped pushing for it after I was able to work around it in rpm-ostree.

[xnox]

Right, I'm working on something similar. I have a set of patches that makes shadow-utils operate with /etc/* files missing (e.g. without /etc/passwd /etc/shadow etc.) and I am starting adding bits to load up nss-altfiles databases in a sensible manner. I'll check your patch to see which other bits I can borrow. Hopefully this can be fixed in shadow-utils and Linux-PAM directly, as imho both should be able to support altfiles.

[cgwalters]

See also https://lists.fedoraproject.org/pipermail/devel/2014-July/200915.html

[cgwalters]

An approach to dealing with setuid binaries in /usr is https://bugzilla.gnome.org/show_bug.cgi?id=722984

[cgwalters]

One thing I discussed with @jlebon today is we realized that for /etc, we end up making physical copies (or reflinks), which means we have an opportunity to chown them on the client machine. We can store the uid for files in /etc in a separate metadata file that ostree processes when doing deployments.

[cgwalters]

Systemd now has support for dynamic users: systemd/systemd@4ffe247
This addresses some of the problem space in that for probably a good chunk of services, they don't need persistent entries in any form.

[dustymabe]

any progress on this?

[cgwalters]

See ostreedev/ostree-releng-scripts#15 (comment) for some interesting data.

One clear bug here is that our tmpfiles.d generation from /var really needs to synthesize back to names, not ids.

[travier]

We could probably start by converting our passwd & group files into sysusers files and ship them. Or add a converter to rpm-ostree to create a sysusers files with hardcoded IDs from the passwd & group entries.

[cgwalters]

Yes, I think nothing stops us today from synthesizing sysusers entries from the stuff in passwd.

I think we could make this extra safe by having it be opt-in and conditional. In fact we already have such a thing in etc-group-members. I think we could switch the implementation of that to be generating a sysusers.d entry for the group instead of injecting it into /etc/group.

[lucab]

I think it is useful to first identify all static UIDs/GIDs, and teach those to rpm-ostree.
Those are the initial simplest cases to handle, as they can be directly lifted into sysusers fragments with their well-known IDs.

A general translation is a little more complex as we have to handle cases like "add user <foo> to group <bar>" which may cross package boundary. This also applies to users/groups which do not own files in the ostree commit. Also, we may (or may not) face issues with lexicographic order of fragments, and the dependencies between their content.

[lucab]

I've found out that the list of static UIDs and GIDs for Fedora/RHEL/etc can be sourced from https://pagure.io/setup/blob/master/f/uidgid.

[cgwalters]

I think going forward we may want to use https://www.freedesktop.org/software/systemd/man/nss-systemd.html#Static%20Drop-In%20JSON%20User/Group%20Records

[cgwalters]

New thread on moving sysusers to rpm https://lists.fedoraproject.org/archives/list/[email protected]/thread/NEFOV236FJYS2RED2SEOV5YHDFLDX7DK/

Bigger picture, I think we can continue to make iterative progress on "thinning out" the data in the static passwd and group databases that are shipped in various places; e.g. I think there's a copy in fedora-iot too and silverblue too.

We need to be careful about this; to start I'd say we should only try dropping entries which have an exact match in sysusers. Offhand it looks like e.g. adm may be a good candidate.

[travier]

To make that happen we need two things:

• Figure out a way to keep existing users as is on current systems that update to a non nss-altfiles image.
• Have new systems populate users/group files from a mix of 'static' /usr/lib/passwd files (for example https://pagure.io/setup/blob/master/f/passwd from https://src.fedoraproject.org/rpms/setup) and sysusers configs on first boot, thus not relying on nss-altfiles anymore.

So for coreos/fedora-coreos-tracker#1524 the goal is not to remove the line from passwd/group but instead to remove nss-altfiles from the image and make sure that the user/group gets created correctly in /etc/passwd & /etc/group.

Systems that update will need to have barrier to go through this migration step before we reset the paswd/group files to the defaults from Fedora.

[cgwalters]

So for coreos/fedora-coreos-tracker#1524 the goal is not to remove the line from passwd/group but instead to remove nss-altfiles from the image

Yes, dropping nss-altfiles would be great. But that's an all-or-nothing thing. Her work will let us drain the duplicate static user/group files that are copied into multiple places in favor of canonicalizing on the bits from the setup package. So it's much more incremental.

[HuijingHei]

Refer to coreos/fedora-coreos-tracker#1524 (comment) (redirect discussion here)

Probably what we need to do first is add a systemd unit which keys off the firstboot and runs systemd-sysusers --root /sysroot before Ignition runs. That I believe would be safe to do now.

IMU, the unit will create group/users (according to configs under /usr/lib/sysusers.d/) in /etc/{group,passwd}, and the users might be already existed in /usr/lib/{group,passwd}, is this expected?

[cgwalters]

I think sysusers should query the nss subsystem which will include altfiles. So sysusers should not create users which are still in the altfiles database.

[HuijingHei]

I think sysusers should query the nss subsystem which will include altfiles. So sysusers should not create users which are still in the altfiles database.

IMU, as non-use of nss-altfiles in the initramfs, by default sysusers can not find users database in /lib.
Look more about coreos/fedora-coreos-config#774, seems need to mount and chroot the target, then sysusers can query the nss subsystem via altfiles, but I might misunderstood.

[cgwalters]

Hmm, yes, the lack of altfiles in the initramfs is potentially a problem. We probably do need to inject it there.

But I didn't think we were running sysusers from the initramfs, but only from the real root?

[HuijingHei]

Hmm, yes, the lack of altfiles in the initramfs is potentially a problem. We probably do need to inject it there.

Sorry, I am confusing about this, if we plan to drop nss-altfiles, why should we inject altfiles in initramfs ? Am I wrong here?

But I didn't think we were running sysusers from the initramfs, but only from the real root?

Yes, we only have bin file systemd-sysusers in initramfs (see ignition-ostree), but do not run from initramfs.

[travier]

More detailed proposal in coreos/fedora-coreos-tracker#155 (comment)

[cgwalters]

Sorry, I am confusing about this, if we plan to drop nss-altfiles, why should we inject altfiles in initramfs ? Am I wrong here?

I'd say it'd be a temporary thing - if it helps us eventually drop it by ensuring that systemd-sysusers works consistently across both the initramfs and real root.