Documentation on how to configure FCOS in a VM using static ip addresses

[r3boot]

Hi,

During my migration from CL to FCOS, the biggest difficulty I ran into was configuring a static ip address using a pre-seeded configuration. For CL, this worked by inserting an interface configuration under /etc/systemd/network/, but since FCOS has standardized on NetworkManager (see coreos/fedora-coreos-tracker#24), I needed to migrate this. The following is a writeup of how I accomplished this, together with a description of my usecase and implementation details. If the project is interested, I can rework this into a pull request.

First off, my usecase. I run a bunch of centos8/kvm hypervisors, running FCOS virtuals, which host (sets of) docker/podman containers. The virtuals are using LVM logical volumes as block devices. Because of coreos/coreos-installer#91, I have a set of scripts which fetches the latest FCOS using a dockerized coreos-installer, and I use dd to write this to the LV. Next, I partprobe this device, mount the first partition, and upload my ignition config to ignition/config.ign, umount everything, generate a KVM configuration and boot up the VM. All this is wrapped in a Makefile, so 'make ' and 5 minutes of time is enough to fully bootstrap a FCOS vm.

To get this setup to work using NetworkManager, I had to change a few bits in my configuration:

• I set a fixed mac address for the VM
• I altered the kernel boot parameters to prevent dracut from requesting a DHCP lease:

sed -i -e 's,rd.neednet=1 ip=dhcp,rd.neednet=0,g' /mnt/grub2/grub.cfg

• I inserted a NM system connection using ignition:

storage:
  files:
    - path: /etc/NetworkManager/system-connections/eth0.nmconnection
      mode: 0600
      contents:
        inline: |
          [connection]
          id=Uplink
          type=802-3-ethernet
          autoconnect=true
          interface-name=eth0
          match-device=interface-name:eth0
          uuid=<randomly generated>
          [ethernet]
          mac-address=<generated mac address for vm>
          [ipv4]
          method=manual
          addresses=192.0.2.1/24
          gateway=192.0.2.255
          dns=192.0.2.53
          dns-search=example.com

After these changes, I was able to boot the VM using a static ip address, without it doing a DHCP request @ boot.

[lucab]

@r3boot thanks for the followup, we'll try to digest these details into a doc-page.

The dracut DHCP is a bit of an unfortunate default for your usecase, but we don't have a good strategy to turn it off declaratively while provisioning.
What happens if you skip the sed part above? Does the node successfully boot after some (possibly large) timeout expiration?

[rbicker]

Hi

We are having a pretty similar use case but a different environment. We use ansible to deploy the Fedora CoreOS OVA to VMware and to set the ignition config as vApp properties. The network configuration is pretty much the same as the one @r3boot posted.

So when we deploy a VM to a network where no DHCP server is available, the server tries to get a dynamic IP (and of course fails). However after some timeout process, the server continues to boot and suceeds in the end, having the static IP assigned to eth0.

So I guess it works without the sed part, it's just wasting a few minutes, so I am wondering if there is a better solution for this scenario? From my understanding, it is not possible to edit the grub config from ignition because the bootloader runs before ignition, right?

Would love to get some input on this, if you would like me to test something I am happy to help.

Thanks!

[lucab]

However after some timeout process, the server continues to boot and succeeds in the end

Thanks for confirming this.

So I guess it works without the sed part, it's just wasting a few minutes, so I am wondering if there is a better solution for this scenario? From my understanding, it is not possible to edit the grub config from ignition because the bootloader runs before ignition, right?

Correct, for the first boot it has no way of changing those. We are looking at https://github.com/coreos/ignition-dracut/issues/81 to make kargs handling a bit easier, but it still won't cancel the very first timeout pause.

For subsequent boots, you can change kernel arguments via rpm-ostree kargs, but that requires fully provisioning first and then rebooting.

[adriannemora]

I started to address this in #30 .

[jlebon]

This is fixed by #30.