diff --git a/README.md b/README.md index a820b49..bfa0daf 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,3 @@ -


kairos-white-column 5bc2fe34 @@ -10,7 +9,6 @@
- Welcome to the community-bundles repository! This repository builds and pushes Kairos community bundles that can be consumed by Kairos core or derivative images (such as [provider-kairos](https://github.com/kairos-io/provider-kairos) ) to extend Kairos configurations and settings, and to add cloud-config keywords. Please note that these community bundles are not officially supported and are provided on a best-effort basis by the community. @@ -36,8 +34,8 @@ To use a community bundle, you can load it with the bundles block in the Kairos ```yaml bundles: -- targets: - - run://quay.io/kairos/community-bundles: + - targets: + - run://quay.io/kairos/community-bundles: ``` Here is an example of how you might use a community bundle in a Kairos core image: @@ -45,20 +43,20 @@ Here is an example of how you might use a community bundle in a Kairos core imag ```yaml #cloud-config install: - device: "auto" - auto: true - reboot: true - image: "docker:quay.io/kairos/kairos-opensuse:v1.4.0-k3sv1.26.0-k3s1" + device: "auto" + auto: true + reboot: true + image: "docker:quay.io/kairos/kairos-opensuse:v1.4.0-k3sv1.26.0-k3s1" users: -- name: "kairos" - passwd: "kairos" - ssh_authorized_keys: - - ... + - name: "kairos" + passwd: "kairos" + ssh_authorized_keys: + - ... bundles: -- targets: - - run://quay.io/kairos/community-bundles:kubevirt + - targets: + - run://quay.io/kairos/community-bundles:kubevirt k3s: enabled: true @@ -77,8 +75,8 @@ To configure the bundle, use the `calico` block: # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:calico_latest + - targets: + - run://quay.io/kairos/community-bundles:calico_latest # Specify calico settings calico: @@ -94,7 +92,7 @@ calico: version: 3.25.0 ``` -Note that specifying `values` and `version` are optional. Specifying `values` allows you to +Note that specifying `values` and `version` are optional. Specifying `values` allows you to [customize the Helm Chart](https://docs.tigera.io/calico/latest/getting-started/kubernetes/helm#customize-the-helm-chart). ### Cert-manager @@ -108,14 +106,45 @@ The bundle does add a `certManager` block, that allow to change the version (cur # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:cert-manager_latest + - targets: + - run://quay.io/kairos/community-bundles:cert-manager_latest # Specify cert-manager settings certManager: version: v1.11.0 ``` +### Flux + +This installs [FluxCD](https://fluxcd.io/flux/cmd/flux_bootstrap/) and supports +automatically bootstrapping the cluster. Only one node will do the bootstrap. +It will time out after trying for 30 minutes and it requires `systemd`. + +```yaml +#cloud-config + +k3s: + enabled: true + +bundles: + - targets: + - run://quay.io/kairos/community-bundles:flux_latest + +# Specify command-line arguments as keys under a key of `bitbucket_server`, +# `git`, `github` or `gitlab` for the provider to boostrap from. An example for +# `github` is shown below. +flux: + env: + # Override default $KUBECONFIG of /etc/rancher/k3s/k3s.yaml if needed + # KUBECONFIG: /home/csagan/.kube/config + GITHUB_TOKEN: abcde1234 + github: + owner: csagan + repository: fleet-infra + path: clusters/cosmos + components-extra: image-reflector-controller,image-automation-controller +``` + ### Kairos The Kairos bundle deploys the [Kairos helm-charts](https://github.com/kairos-io/helm-charts). It installs the `kairos-crds` chart, and allows to enable [entangle-proxy](https://kairos.io/docs/reference/entangle/), [osbuilder](https://kairos.io/docs/advanced/build/), and [entangle](https://kairos.io/docs/reference/entangle/). @@ -127,8 +156,8 @@ By default the bundle will install only the CRDs, components needs to be explici # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:kairos_latest + - targets: + - run://quay.io/kairos/community-bundles:kairos_latest # Specify kairos bundle setting kairos: @@ -154,24 +183,22 @@ To configure the bundle, use the `kyverno` block: # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:kyverno_latest + - targets: + - run://quay.io/kairos/community-bundles:kyverno_latest # Specify kyverno settings kyverno: - values: - .... + values: .... version: ... ``` -Note that specifying `values` and `version` are optional. Specifying `values` allows you to +Note that specifying `values` and `version` are optional. Specifying `values` allows you to [customize the Helm Chart](https://github.com/kyverno/kyverno/blob/main/charts/kyverno/values.yaml). ### Kubevirt The Kubevirt bundle deploys [Kubevirt](https://github.com/kubevirt/kubevirt) and optionally [kubevirt-manager](https://kubevirt-manager.io/) - The bundle does add a `kubevirt` block, that allow to enable `kubevirt-manager`: ```yaml @@ -179,8 +206,8 @@ The bundle does add a `kubevirt` block, that allow to enable `kubevirt-manager`: # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:kubevirt_latest + - targets: + - run://quay.io/kairos/community-bundles:kubevirt_latest # Specify kubevirt settings kubevirt: @@ -198,8 +225,8 @@ To configure the bundle, use the `longhorn` block: # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:longhorn_latest + - targets: + - run://quay.io/kairos/community-bundles:longhorn_latest # Specify longhorn settings longhorn: @@ -209,7 +236,7 @@ longhorn: version: 1.4.0 ``` -Note that specifying `values` and `version` are optional. Specifying `values` allows you to +Note that specifying `values` and `version` are optional. Specifying `values` allows you to [customize the Helm Chart](https://longhorn.io/docs/latest/advanced-resources/deploy/customizing-default-settings/#using-helm). ### MetalLB @@ -223,8 +250,8 @@ The bundle does add a `metallb` block, that allow to set up the MetalLB version # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:metallb_latest + - targets: + - run://quay.io/kairos/community-bundles:metallb_latest # Specify metallb settings metallb: @@ -239,20 +266,20 @@ Note, you might want to disable the default LoadBalancer of k3s, a full example hostname: kairoslab-{{ trunc 4 .MachineID }} users: -- name: kairos - ssh_authorized_keys: - # Add your github user here! - - github:mudler + - name: kairos + ssh_authorized_keys: + # Add your github user here! + - github:mudler k3s: enable: true args: - - --disable=servicelb + - --disable=servicelb # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:metallb_latest + - targets: + - run://quay.io/kairos/community-bundles:metallb_latest # Specify metallb settings metallb: @@ -271,8 +298,8 @@ To configure the bundle, use the `multus` block: ```yaml # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:multus_latest + - targets: + - run://quay.io/kairos/community-bundles:multus_latest # Specify multus settings. Here are the defaults: multus: @@ -326,8 +353,8 @@ The bundle does add a `suc` block, that allow to change the version: # Specify the bundle to use bundles: -- targets: - - run://quay.io/kairos/community-bundles:system-upgrade-controller_latest + - targets: + - run://quay.io/kairos/community-bundles:system-upgrade-controller_latest # Specify system-upgrade-controller settings suc: diff --git a/flux/Dockerfile b/flux/Dockerfile new file mode 100644 index 0000000..a60d2da --- /dev/null +++ b/flux/Dockerfile @@ -0,0 +1,19 @@ +FROM alpine as build +ENV VERSION=2.1.2 +ENV CHECKSUM=61b360b50d6cfc34410730b1cebeb75f5eda2b484e47b9a083412f51ad56de68 + +ADD https://github.com/fluxcd/flux2/releases/download/v${VERSION}/flux_${VERSION}_linux_amd64.tar.gz /tmp +RUN DOWNLOAD_FILE="/tmp/flux_${VERSION}_linux_amd64.tar.gz" && \ + DOWNLOAD_CHECKSUM=$(sha256sum "${DOWNLOAD_FILE}" | awk '{print $1}') && \ + if [[ ${DOWNLOAD_CHECKSUM} != ${CHECKSUM} ]]; then \ + echo "Checksum does not match"; \ + exit 1; \ + fi && \ + tar xzf "${DOWNLOAD_FILE}" -C / && \ + rm "${DOWNLOAD_FILE}" + +FROM scratch +COPY --from=build flux . +COPY flux-bootstrap.service . +COPY flux-bootstrap.sh . +COPY run.sh . diff --git a/flux/flux-bootstrap.service b/flux/flux-bootstrap.service new file mode 100644 index 0000000..3526df3 --- /dev/null +++ b/flux/flux-bootstrap.service @@ -0,0 +1,15 @@ +[Unit] +Description=Bootstrap cluster with Flux +Documentation=https://github.com/kairos-io/community-bundles/blob/main/README.md#flux +After=k3s.service + +[Service] +Type=oneshot +Restart=no +ExecStart=/usr/local/bin/flux-bootstrap.sh +User=root +Group=root +RemainAfterExit=yes + +[Install] +WantedBy=k3s.service diff --git a/flux/flux-bootstrap.sh b/flux/flux-bootstrap.sh new file mode 100755 index 0000000..2910000 --- /dev/null +++ b/flux/flux-bootstrap.sh @@ -0,0 +1,120 @@ +#!/bin/bash + +# Override with the `flux.env.KUBECONFIG` key if necessary +export KUBECONFIG=/etc/rancher/k3s/k3s.yaml + +# Seconds +short=5 +long=900 + +configmap=flux-bootstrap + +info () { + echo $'\e[36mINFO\e[0m: ' "$1" +} + +warn() { + echo $'\e[33mWARN\e[0m:' "$1" +} + +error() { + echo $'\e[31mERROR\e[0m:' "$1" +} + +cleanup() { + info "Removing bootstrap configmap" + timeout $short kubectl delete configmap -n default $configmap +} + +# Don't bootstrap if the cluster is already bootstrapped +if flux version &>/dev/null; then + info "Flux is already bootstrapped, exiting..." + exit 0 +fi + +# Determine what VCS we need to bootstrap +for vcs in bitbucket_server git github gitlab; do + if [[ $(kairos-agent config get flux.$vcs 2>/dev/null) != "null" ]]; then + version_control=$vcs + break + fi +done + +if [[ "${version_control}x" == "x" ]]; then + error "Unable to determine what version control provider to use, exiting..." + exit 1 +fi + +# Get flux envs and settings for our VCS +mapfile -t envs < <(kairos-agent config get "flux.env" 2>/dev/null) +mapfile -t args < <(kairos-agent config get "flux.$version_control" 2>/dev/null) +declare -a cmdline + +for setting in "${envs[@]}"; do + if [[ $setting != "null" ]]; then + env=$(echo "$setting" | cut -d: -f1) + value=$(echo "$setting" | cut -d':' -f2 | sed -E 's/^ //g') + if [[ "${value}x" != "x" ]]; then + export "$env"="$value" + fi + fi +done + +# Set commandline args +for setting in "${args[@]}"; do + if [[ $setting != "null" ]]; then + arg=$(echo "$setting" | cut -d: -f1) + value=$(echo "$setting" | cut -d':' -f2 | sed -E 's/^ //g') + if [[ "${value}x" != "x" ]]; then + cmdline+=("--$arg" "$value") + fi + fi +done + +# Try to bootstrap Flux for 30 minutes, sleep 15 seconds between attempts +minutes=30 +sleep=15 +retry_attempt=1 +total_attempts=$(( minutes * 60 / sleep )) +active="false" + +if [[ "${#cmdline[@]}" -eq 0 ]]; then + info "Flux was not configured in cloud-config, not bootstrapping" + exit 0 +else + while [[ $retry_attempt -le $total_attempts ]]; do + if [[ "$active" != "true" ]]; then + # Ensure only one host tries to bootstrap, whichever makes the configmap first + if ! timeout $short kubectl version &> /dev/null; then + info "Kubernetes API not ready yet, sleeping" + else + if ! timeout $short kubectl create configmap $configmap --from-literal=hostname="$(hostname)"; then + warn "Unable to create configmap, another node may be active" + fi + + # The configmap exists but we must finally check if the hostname matches + if [[ "$(timeout $short kubectl get configmap -n default $configmap -o jsonpath='{.data.hostname}')" != "$(hostname)" ]]; then + error "Flux bootstrap ConfigMap exists but another node is active, exiting..." + exit 3 + fi + + # We must be the active node + active="true" + fi + fi + + if [[ "$active" == "true" ]]; then + if timeout $long flux bootstrap "${version_control/_/-}" "${cmdline[@]}"; then + cleanup + exit 0 + fi + fi + + warn "Install attempt $retry_attempt (of $total_attempts) failed, retrying in $sleep seconds" + (( retry_attempt = retry_attempt + 1 )) + sleep $sleep + done +fi + +error "Failed to bootstrap with Flux, timed out ($minutes minutes)" +exit 4 \ No newline at end of file diff --git a/flux/run.sh b/flux/run.sh new file mode 100755 index 0000000..02112d5 --- /dev/null +++ b/flux/run.sh @@ -0,0 +1,11 @@ +#!/bin/bash +set -x + +bin=/usr/local/bin/ +system=/etc/systemd/system/ + +mkdir -p "$bin" +cp flux "$bin" +cp flux-bootstrap.sh "$bin" +cp flux-bootstrap.service "$system" +systemctl enable flux-bootstrap \ No newline at end of file