Configurability for automated installs without kargs

[jlebon]

This came up during review of #119.

There's some interest there in expanding the kargs functionality. I'd like to make sure this is the path we want to go down, rather than rework the UX to something friendlier.

I know there's some tension here, since we kinda have to keep supporting the base kargs the legacy installer did for backwards compatibility reasons.

OTOH, now that the installer is a simple binary written in a sane language, I think it'd be pretty sweet to have e.g. a YAML file or some other input file that can be dropped in via Ignition (and/or injected into the ISO) which is read directly by coreos-installer.

Then, the legacy kargs compat path would just be translating the kargs into this YAML file. We could also support a single karg, which would be just a URL to the YAML file to fetch.

[jlebon]

(The idea is that it encourages a workflow where this file would be checked into an SCM, changes are audited, reviewed, etc... and installs are fully reproducible.)

[dustymabe]

I think the kargs approach is a nice MVP for an installer, but we obviously don't want a million options, which is part of the reason we now run in the real root. i.e. people can use kargs for simple installs, but if they need something complicated they can use an ignition config to set up the environment and then call the installer.

[lucab]

My personal understanding (and preference) is that we slowly move away from kargs for all cases (both simpler and more complex).

The rationale is the one mentioned by @jlebon. The final goal looks like this: https://github.com/coreos/tectonic-installer/blob/track-1/platforms/metal/cl/coreos-install.yaml.tmpl.

For FCOS specifically, I'd jump straight to document the "real rootfs" usage.

[jlebon]

My personal understanding (and preference) is that we slowly move away from kargs for all cases (both simpler and more complex).

+1

This is why I'm leaning towards a simpler approach which doesn't require everyone writing down systemd units and scripts that call the installer. IOW:

• simple/vanilla installs can just drop a file in a specific location (e.g. /etc/coreos-installer.yaml)
• one can order pre/post-install tasks by just having their units ordered wrt coreos-installer.service
• more complex workflows can just call coreos-installer directly
• keep kargs as the legacy interface

Anyway, to re-iterate I don't think we need to solve this right now since the "call coreos-installer directly" approach should work for now.

[dustymabe]

This definitely needs some dedicated discussion. Can we go with what we have in #119 for now (i.e. resolve this discussion) and pick this up soon after?

[lucab]

Yes, let's not block on this.
We can split the YAML thing to its own ticket (not a huge fan of that, but not strongly against).

[bgilbert]

AIUI the proposal is roughly:

• kargs for legacy or really simple cases (maybe as simple as install_dev=)
• Fully generic Ignition config for really complex cases
• Ignition config writing YAML DSL for intermediate cases, to reduce boilerplate

coreos-installer already has a DSL -- its command line -- and I'm hesitant to introduce a redundant one. Why not provide a way to override the coreos-installer command line invoked by the default systemd service? In the past we've used systemd environment files for this, but since the default service invokes a wrapper script, other mechanisms would also work.

[dustymabe]

AIUI the proposal is roughly:

• kargs for legacy or really simple cases (maybe as simple as install_dev=)
• Fully generic Ignition config for really complex cases
• Ignition config writing YAML DSL for intermediate cases, to reduce boilerplate

I think you're right but I think @jlebon had an emphasis on "let's not add to and eventually not support kargs".

coreos-installer already has a DSL -- its command line -- and I'm hesitant to introduce a redundant one. Why not provide a way to override the coreos-installer command line invoked by the default systemd service? In the past we've used systemd environment files for this, but since the default service invokes a wrapper script, other mechanisms would also work.

I think I agree with you here. I'd rather keep a relatively simple karg interface like what we have and allow for people to easily do shenanigans via "Fully generic Ignition config for really complex cases". I don't anticipate coreos-installer developing a ton of options/arguments so there probably won't even be a ton of lost fidelity between the kargs and the actual CLI.

[bgilbert]

I'd rather keep a relatively simple karg interface like what we have and allow for people to easily do shenanigans via "Fully generic Ignition config for really complex cases". I don't anticipate coreos-installer developing a ton of options/arguments so there probably won't even be a ton of lost fidelity between the kargs and the actual CLI.

To be clear, I'm arguing for a middle ground, where an Ignition config could write e.g. /etc/coreos/installer-args containing -p vmware --stream-base-url [...]. That way it wouldn't need to override or add a drop-in for the default installer service, while still having more flexibility than allowed by kargs.

[dustymabe]

To be clear, I'm arguing for a middle ground, where an Ignition config could write e.g. /etc/coreos/installer-args containing -p vmware --stream-base-url [...]. That way it wouldn't need to override or add a drop-in for the default installer service, while still having more flexibility than allowed by kargs.

Sounds good to me too.

[jlebon]

This is related to openshift/enhancements#492, which suggests a tool to make it easier to customize installs. We want that tool to make use of simple established primitives instead of having to e.g. override the coreos-installer service unit.

[cgwalters]

To be clear, I'm arguing for a middle ground, where an Ignition config could write e.g. /etc/coreos/installer-args containing -p vmware --stream-base-url [...].

I like this one, simple and easy to understand and doesn't really create any new interfaces.

EDIT: Though one thing that is annoying about this is defining the file format; e.g. if any of the arguments need to have spaces or shell metacharacters we need to declare whether those should be escaped or not.

[bgilbert]

We now have install config files, which are YAML files with keys corresponding to coreos-installer install long options. If any are placed in /etc/coreos/installer.d, coreos-installer will automatically run at startup and reboot afterward.

We also have coreos-installer iso/pxe customize for building custom live media that invokes coreos-installer. Those commands can take an install config file, and/or can synthesize one in simple cases.

Closing this out.