From b8b3bfbb2fe28e8fc8a2cafbcdcdc9020489ba05 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Wed, 15 Mar 2023 13:31:07 +0100 Subject: [PATCH] :book: Add documentation on how to gate upgrades with kyverno (#1135) * :book: Add documentation on how to gate upgrades with kyverno Signed-off-by: mudler * :book: Update internal components list Signed-off-by: mudler --------- Signed-off-by: mudler --- docs/content/en/docs/Architecture/meta.md | 18 +++++- docs/content/en/docs/Reference/cli.md | 4 -- .../en/docs/Reference/recovery_mode.md | 2 +- .../en/docs/Reference/troubleshooting.md | 1 - docs/content/en/docs/Upgrade/kubernetes.md | 64 +++++++++++++++++++ tests/assets/autoinstall.yaml | 1 - 6 files changed, 80 insertions(+), 10 deletions(-) diff --git a/docs/content/en/docs/Architecture/meta.md b/docs/content/en/docs/Architecture/meta.md index f294b8a7f..0b15e122c 100644 --- a/docs/content/en/docs/Architecture/meta.md +++ b/docs/content/en/docs/Architecture/meta.md @@ -22,9 +22,21 @@ Every component is extensible and modular such as it can be customized and repla ### Internal components -Kairos encompasses several components, some externally, most notably: - +Kairos encompasses several components, external and internal. + +Internal: +- [kairos](https://github.com/kairos-io/kairos) is the main repository, building the `kairos-agent` and containing the image definitions which runs on our CI pipelines. +- [immucore](https://github.com/kairos-io/immucore) is the immutability management interface. +- [AuroraBoot](https://github.com/kairos-io/AuroraBoot) is the Kairos Node bootstrapper +- [elemental-cli](https://github.com/kairos-io/elemental-cli) manages the installation, reset, and upgrade of the Kairos node. +- [system packages](https://github.com/kairos-io/packages) contains additional packages, cross-distro, partly used in framework images +- [kcrypt](https://github.com/kairos-io/kcrypt) is the component responsible for encryption and decryption of data at rest +- [kcrypt-challenger](https://github.com/kairos-io/kcrypt-challenger) is the `kairos` plugin that works with the TPM chip to unlock LUKS partitions +- [osbuilder](https://github.com/kairos-io/osbuilder) is used to build bootable artifacts from container images +- [entangle](https://github.com/kairos-io/entangle) a CRD to interconnect Kubernetes clusters +- [entangle-proxy](https://github.com/kairos-io/entangle-proxy) a CRD to control interconnetted clusters + +External: - [K3s](https://k3s.io) as a Kubernetes distribution - [edgevpn](https://mudler.github.io/edgevpn) (optional) as fabric for the distributed network, node coordination and bootstrap. Provides also embedded DNS capabilities for the cluster. Internally uses [libp2p](https://github.com/libp2p/go-libp2p) for the P2P mesh capabilities. -- [elemental-toolkit](https://rancher.github.io/elemental-toolkit/docs/) as a fundament to build the Linux derivative. Indeed, any `Elemental` docs applies to `Kairos` as well. - [nohang](https://github.com/hakavlad/nohang) A sophisticated low memory handler for Linux. diff --git a/docs/content/en/docs/Reference/cli.md b/docs/content/en/docs/Reference/cli.md index 83213c33c..ab84e683c 100644 --- a/docs/content/en/docs/Reference/cli.md +++ b/docs/content/en/docs/Reference/cli.md @@ -61,8 +61,6 @@ kairos: device: "" poweroff: false -# Cloud init syntax to setup users. -# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/ stages: network: - name: "Setup users" @@ -90,8 +88,6 @@ kairos: device: "" poweroff: false -# Cloud init syntax to setup users. -# See https://rancher.github.io/elemental-toolkit/docs/reference/cloud_init/ stages: network: - name: "Setup users" diff --git a/docs/content/en/docs/Reference/recovery_mode.md b/docs/content/en/docs/Reference/recovery_mode.md index 3bdf3a562..e66232ad1 100644 --- a/docs/content/en/docs/Reference/recovery_mode.md +++ b/docs/content/en/docs/Reference/recovery_mode.md @@ -9,7 +9,7 @@ description: > The Kairos recovery mode can be used to recover a damaged system or to regain access remotely (with assistance) to a machine which has been lost access to. The recovery mode is accessible only from the GRUB menu, from both the LiveCD, and an installed system. {{% alert title="Note" %}} -On installed system, there are two recovery modes available during boot. Below describes only how the Kairos remote recovery works. The manual recovery entry has nothing special from the standard Elemental-toolkit recovery mode. It can be used to reset the A/B partitions (with the user/pass used during setup) and perform any other operation without remote access. +On installed system, there are two recovery modes available during boot. Below describes only how the Kairos remote recovery works. It can be used to reset the A/B partitions (with the user/pass used during setup) and perform any other operation without remote access. {{% /alert %}} ## Boot into recovery mode diff --git a/docs/content/en/docs/Reference/troubleshooting.md b/docs/content/en/docs/Reference/troubleshooting.md index 8b2af731a..f308aaf74 100644 --- a/docs/content/en/docs/Reference/troubleshooting.md +++ b/docs/content/en/docs/Reference/troubleshooting.md @@ -42,4 +42,3 @@ On all nodes, which are deployed with the P2P full-mesh feature of the cluster, ## See also - [Dracut debug docs](https://fedoraproject.org/wiki/How_to_debug_Dracut_problems) -- [Elemental troubleshooting docs](https://rancher.github.io/elemental-toolkit/docs/reference/troubleshooting/) diff --git a/docs/content/en/docs/Upgrade/kubernetes.md b/docs/content/en/docs/Upgrade/kubernetes.md index 6d46d9d46..17d80eb2b 100644 --- a/docs/content/en/docs/Upgrade/kubernetes.md +++ b/docs/content/en/docs/Upgrade/kubernetes.md @@ -72,6 +72,70 @@ system-upgrade apply-os-upgrade-on-kairos-with-1a1a24bcf897bd275730bdd8548-h7f Done! We should have all the basics to get our first cluster rolling, but there is much more we can do. +## Verify images attestation during upgrades + +Container images can be signed during the build phase of a CI/CD pipeline using [Cosign](https://github.com/sigstore/cosign), Kairos signs every artifact as part of the release process. + +To ensure that the images used during upgrades match the expected signatures, [Kyverno](https://kyverno.io/) can be used to set up policies. This is done by checking if the signature is present in the OCI registry and if the image was signed using the specified key. The policy rule check fails if either of these conditions is not met. + +To learn more about this specific Kyverno feature, you can refer to the [documentation](https://kyverno.io/docs/writing-policies/verify-images/). This allows for the verification of image authenticity directly at the node level prior to upgrading. + +A Kyverno policy for `provider-kairos` images might look like the following: + +```yaml +apiVersion: kyverno.io/v1 +kind: ClusterPolicy +metadata: + name: check-image +spec: + validationFailureAction: Enforce + background: false + webhookTimeoutSeconds: 30 + failurePolicy: Fail + rules: + - name: check-image + match: + any: + - resources: + kinds: + - Pod + verifyImages: + - imageReferences: + - "quay.io/kairos/kairos-*" + attestors: + - entries: + # See: https://kyverno.io/docs/writing-policies/verify-images/#keyless-signing-and-verification + - keyless: + subject: "https://github.com/kairos-io/provider-kairos/.github/workflows/release.yaml@refs/tags/*" + issuer: "https://token.actions.githubusercontent.com" + rekor: + url: https://rekor.sigstore.dev +``` + +To install Kyverno in a Kairos cluster, you can simply use the community [bundles](/docs/advanced/bundles). For example, you can use the following installation cloud config file: + +```yaml +#cloud-config + +hostname: kyverno-{{ trunc 4 .MachineID }} + +# Specify the bundle to use +bundles: +- targets: + - run://quay.io/kairos/community-bundles:system-upgrade-controller_latest + - run://quay.io/kairos/community-bundles:cert-manager_latest + - run://quay.io/kairos/community-bundles:kyverno_latest + +users: +- name: kairos + passwd: kairos + +k3s: + enabled: true +``` + +This configuration file prepare the system with the `cert-manager`, `system-upgrade-controller` and the `kyverno` bundle, enabling `k3s`. + ## Customize the upgrade plan It is possible to run additional commands before the upgrade takes place into the node, consider the following example: diff --git a/tests/assets/autoinstall.yaml b/tests/assets/autoinstall.yaml index 8247d351b..e84743952 100644 --- a/tests/assets/autoinstall.yaml +++ b/tests/assets/autoinstall.yaml @@ -4,7 +4,6 @@ install: auto: true reboot: true device: /dev/vda - # https://rancher.github.io/elemental-toolkit/docs/customizing/configure_grub/#grub-environment-variables grub_options: extra_cmdline: "foobarzz"