add coreos-teardown-initramfs-network.service #159

dustymabe · 2020-03-19T20:46:14Z

This is a forward port of coreos-teardown-initramfs-network.service
from the spec2x branch [1] (used for RHEL CoreOS). When moving to NM
in the initrd [2] we decided that we also needed a mechanism to take down
the networking between the initramfs and the real root. While we would
like to use NetworkManager's logic to do this operation in the future
it's currently not easily achieved because NetworkManager is not running
persistently in the initramfs [3].

Additionally, we have decided to propagate initramfs networking configuration
in some scenarios [4]. The policy here is:

If a networking configuration was provided before this point
(most likely via Ignition) and exists in the real root then
we do nothing and don't propagate any initramfs networking.
If a user did not provide any networking configuration
then we'll propagate the initramfs networking configuration
into the real root.

[1] #78
[2] coreos/fedora-coreos-tracker#394
[3] https://bugzilla.redhat.com/show_bug.cgi?id=1814038
[4] coreos/fedora-coreos-tracker#394 (comment)

dustymabe · 2020-03-19T20:53:46Z

this is working in initial tests but I need to look at things more comprehensively before we merge this.. setting to draft for now

cgwalters · 2020-03-19T20:53:59Z

I think one reason we weren't doing this in FCOS is that we ended up with two DHCP leases or something. We need to also kill dhclient; maybe it's watching for the interface going down, but that's going to be racy because it may not have a chance to release the lease before being killed by switchroot.

dustymabe · 2020-03-19T21:02:45Z

@cgwalters - I haven't seen any issues with that in initial tests with NM in the initrd. I'll watch out for it.

jlebon · 2020-03-19T21:06:38Z

I think the double lease issue was fixed since coreos/fedora-coreos-config#82.

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

jlebon · 2020-03-20T15:24:35Z

Could we split the tearing down part from the persisting part? Presumably once we can use NM for the former, that service would just be e.g. ExecStart=nmcli networking off, right?

dustymabe · 2020-03-20T19:26:55Z

Could we split the tearing down part from the persisting part? Presumably once we can use NM for the former, that service would just be e.g. ExecStart=nmcli networking off, right?

right.

dustymabe · 2020-03-20T19:46:57Z

though one problem with doing that is the question of who is responsible for removing /run/NetworkManager/* files. Is it the teardown or the propagate.

dustymabe · 2020-03-24T03:42:46Z

ok I talked with jlebon and we are going to leave this in a single teardown script/service for now.

I did a bunch of testing on this using a script to simulate different scenarios. It seems like things are going well so far. This script is at https://dustymabe.fedorapeople.org/fcos-network-testing.sh . I need to find the right place to put it for now. I've been told that network testing in our CI is going to be somewhat limited for now because of unpriv qemu networking but maybe we can use this script as inspiration for CI tests when that is no longer a blocker.

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

darkmuggle

A couple of questions, possibly nits.

dracut/30ignition/coreos-teardown-initramfs-network.sh

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

darkmuggle · 2020-03-24T19:57:25Z

LGTM, I guess my comments were nits.

dracut/30ignition/module-setup.sh

dracut/30ignition/ignition-generator

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. We effectively move *to* NetworkManager in the initrd (the default in Fedora 31+) by putting back in the nm-initrd-generator, which we were previously removing. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

This is a forward port of coreos-teardown-initramfs-network.service from the spec2x branch [1] (used for RHEL CoreOS). When moving to NM in the initrd [2] we decided that we also needed a mechanism to take down the networking between the initramfs and the real root. While we would like to use NetworkManager's logic to do this operation in the future it's currently not easily achieved because NetworkManager is not running persistently in the initramfs [3]. [1] coreos#78 [2] coreos/fedora-coreos-tracker#394 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1814038

This will make it easier for us to add a bit more code into this file.

…g if desired The policy here is: - If a networking configuration was provided before this point (most likely via Ignition) and exists in the real root then we do nothing and don't propagate any initramfs networking. - If a user did not provide any networking configuration then we'll propagate the initramfs networking configuration into the real root. See coreos/fedora-coreos-tracker#394 (comment)

Remove the removal of the `/tmp/net.$1.did-setup` (used by network-legacy dracut module) and start removing files under /run/NetworkManager/ instead.

down_teams() is a placeholder for now to remind us that teams do exist and need to be considered. If we find things that need to be done then we'll add actions to do in the down_teams() function in the future.

On recommendation from the NM team let's take down routes as well.

On recommendation from the NM team let's try to delete the device first and if that doesn't work then set it to down and flush any associated addresses. Deleting virtual devices (bonds, teams, bridges, ip-tunnels, etc) will clean up any associated kernel resources, so this makes the logic for bringing down these devices easier. A real device can't be deleted so that will fail and we'll fallback to setting it down and flushing addresses.

Since we have ignition-complete.target we don't need to use the generator to enable the unit.

jlebon

LGTM!

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. We effectively move *to* NetworkManager in the initrd (the default in Fedora 31+) by putting back in the nm-initrd-generator, which we were previously removing. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

Needed for appropriately implementing NetworkManager support in the initramfs. See coreos/ignition-dracut#159

dustymabe · 2020-03-24T22:17:35Z

FYI: I've been testing this using an ugly script for now (better than hand testing at least 😄). It goes through and makes sure variations of initrd passed networking config and ignition passed networking config works. See coreos/fedora-coreos-config#316

Moving to NetworkManager in the initrd should help us solve some problems we've been having with Networking. For what we want to do in Fedora CoreOS doing this right requires the network to be torn down in the initrd and also possibly propagated forward. We effectively move *to* NetworkManager in the initrd (the default in Fedora 31+) by putting back in the nm-initrd-generator, which we were previously removing. Requires: coreos/ignition-dracut#159 Fixes: coreos/fedora-coreos-tracker#394

Needed for appropriately implementing NetworkManager support in the initramfs. See coreos/ignition-dracut#159

This is a backport of the commits merged in f67d587, which was part of coreos#159. The upstream commits persisted Networking as part of the teardown service so we also remove `persist-ifcfg.sh`.

This is a backport of the merge commits f67d587 (coreos#159), and 390779d (coreos#174). The upstream commits persisted networking configuration and hostname as part of the teardown service so we also remove `persist-ifcfg.sh`.

dustymabe changed the title ~~add coreos-teardown-initramfs.service~~ add coreos-teardown-initramfs-network.service Mar 19, 2020

dustymabe force-pushed the dusty-network branch from 268ceb4 to aaadaca Compare March 19, 2020 20:49

dustymabe mentioned this pull request Mar 19, 2020

fedora-coreos-base: use NetworkManager for networking in the initramfs coreos/fedora-coreos-config#310

Merged

jlebon mentioned this pull request Mar 20, 2020

static IP configured via NM keyfile is not respected on first boot coreos/fedora-coreos-tracker#358

Closed

This was referenced Mar 23, 2020

DNS issues on Azure coreos/fedora-coreos-tracker#356

Closed

Using NetworkManager in the initramfs coreos/fedora-coreos-tracker#394

Closed

dustymabe marked this pull request as ready for review March 24, 2020 03:42

darkmuggle reviewed Mar 24, 2020

View reviewed changes

dracut/30ignition/coreos-teardown-initramfs-network.sh Show resolved Hide resolved

dracut/30ignition/coreos-teardown-initramfs-network.sh Show resolved Hide resolved

dustymabe force-pushed the dusty-network branch 2 times, most recently from 46726e4 to 0e3e188 Compare March 24, 2020 19:09

jlebon reviewed Mar 24, 2020

View reviewed changes

dracut/30ignition/module-setup.sh Outdated Show resolved Hide resolved

dracut/30ignition/ignition-generator Outdated Show resolved Hide resolved

dustymabe added 6 commits March 24, 2020 16:35

coreos-teardown-initramfs-network: refactor into more functions

70c7cd5

This will make it easier for us to add a bit more code into this file.

coreos-teardown-initramfs-network: remove state files on switch

e0efb09

Remove the removal of the `/tmp/net.$1.did-setup` (used by network-legacy dracut module) and start removing files under /run/NetworkManager/ instead.

coreos-teardown-initramfs-network: take down teams as well

42b2f06

down_teams() is a placeholder for now to remind us that teams do exist and need to be considered. If we find things that need to be done then we'll add actions to do in the down_teams() function in the future.

coreos-teardown-initramfs-network: take down routes too

03b0035

On recommendation from the NM team let's take down routes as well.

dustymabe added 2 commits March 24, 2020 16:35

coreos-teardown-initramfs-network: enable the service statically

89f2673

Since we have ignition-complete.target we don't need to use the generator to enable the unit.

dustymabe force-pushed the dusty-network branch from 0e3e188 to 89f2673 Compare March 24, 2020 20:42

jlebon reviewed Mar 24, 2020

View reviewed changes

dustymabe merged commit f67d587 into coreos:master Mar 24, 2020

dustymabe added a commit to dustymabe/fedora-coreos-config that referenced this pull request Mar 24, 2020

overrides: fasttrack ignition-2.2.1-1.git2d3ff58.fc31.x86_64

8cb6b8c

Needed for appropriately implementing NetworkManager support in the initramfs. See coreos/ignition-dracut#159

dustymabe added a commit to coreos/fedora-coreos-config that referenced this pull request Mar 24, 2020

overrides: fasttrack ignition-2.2.1-1.git2d3ff58.fc31.x86_64

07a1be3

Needed for appropriately implementing NetworkManager support in the initramfs. See coreos/ignition-dracut#159

dustymabe mentioned this pull request Mar 24, 2020

[spec2x] coreos-teardown-initramfs-network: evaluate necessity of the script #119

Closed

bgilbert mentioned this pull request Mar 29, 2020

packet: Don't report umount stage status to Packet control plane coreos/ignition#952

Closed

dustymabe mentioned this pull request Mar 30, 2020

Regression in gcp, cant fetch metadata coreos/fedora-coreos-tracker#440

Closed

dustymabe deleted the dusty-network branch April 21, 2020 20:46

dustymabe mentioned this pull request Apr 24, 2020

[spec2x] backport coreos-teardown-initramfs-network.service #175

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add coreos-teardown-initramfs-network.service #159

add coreos-teardown-initramfs-network.service #159

dustymabe commented Mar 19, 2020

dustymabe commented Mar 19, 2020

cgwalters commented Mar 19, 2020

dustymabe commented Mar 19, 2020

jlebon commented Mar 19, 2020

jlebon commented Mar 20, 2020 •

edited

Loading

dustymabe commented Mar 20, 2020

dustymabe commented Mar 20, 2020

dustymabe commented Mar 24, 2020

darkmuggle left a comment

darkmuggle commented Mar 24, 2020 •

edited

Loading

jlebon left a comment

dustymabe commented Mar 24, 2020

add coreos-teardown-initramfs-network.service #159

add coreos-teardown-initramfs-network.service #159

Conversation

dustymabe commented Mar 19, 2020

dustymabe commented Mar 19, 2020

cgwalters commented Mar 19, 2020

dustymabe commented Mar 19, 2020

jlebon commented Mar 19, 2020

jlebon commented Mar 20, 2020 • edited Loading

dustymabe commented Mar 20, 2020

dustymabe commented Mar 20, 2020

dustymabe commented Mar 24, 2020

darkmuggle left a comment

Choose a reason for hiding this comment

darkmuggle commented Mar 24, 2020 • edited Loading

jlebon left a comment

Choose a reason for hiding this comment

dustymabe commented Mar 24, 2020

jlebon commented Mar 20, 2020 •

edited

Loading

darkmuggle commented Mar 24, 2020 •

edited

Loading