The "server:" URL output by the "k0sctl kubeconfig" command is not a CPLB IP address.

[chattytak]

If CPLB is enabled, when kubeconfig is generated with the k0sctl kubeconfig command,
server: URL will be the real IP address of the controller, not the IP address of the CPLB.

The only way to get rid of this problem is to add an option like k0sctl kubeconfig --address https://CPLBIP:6443.
Is this the intended specification?

# k0sctl kubeconfig --debug
DEBU[0000] Loaded configuration:
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - ssh:
      address: 192.168.0.2
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: controller
    installFlags:
    - --enable-metrics-scraper
  - ssh:
      address: 192.168.0.3
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: controller
    installFlags:
    - --enable-metrics-scraper
  - ssh:
      address: 192.168.0.4
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: controller
    installFlags:
    - --enable-metrics-scraper
  - ssh:
      address: 192.168.0.18
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: worker
  - ssh:
      address: 192.168.0.19
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: worker
  - ssh:
      address: 192.168.0.20
      user: root
      port: 22
      keyPath: ~/.ssh/id_rsa
    uploadBinary: true
    role: worker
  k0s:
    version: v1.31.1+k0s.1
    versionChannel: stable
    dynamicConfig: false
    config:
      apiVersion: k0s.k0sproject.io/v1beta1
      kind: ClusterConfig
      metadata:
        name: k0s
      spec:
        api:
          k0sApiPort: 9443
          port: 6443
          sans:
          - 192.168.0.10
          extraArgs:
            default-not-ready-toleration-seconds: "30"
            default-unreachable-toleration-seconds: "30"
        installConfig:
          users:
            etcdUser: etcd
            kineUser: kube-apiserver
            konnectivityUser: konnectivity-server
            kubeAPIserverUser: kube-apiserver
            kubeSchedulerUser: kube-scheduler
        konnectivity:
          adminPort: 8133
          agentPort: 8132
        network:
          calico: null
          clusterDomain: cluster.local
          dualStack: {}
          kubeProxy:
            disabled: true
          podCIDR: 10.244.0.0/16
          provider: custom
          serviceCIDR: 10.96.0.0/12
          controlPlaneLoadBalancing:
            enabled: true
            type: Keepalived
            keepalived:
              vrrpInstances:
              - virtualIPs: ["192.168.0.10/24"]
                authPass: CPLB
              virtualServers:
              - ipAddress: 192.168.0.10
          nodeLocalLoadBalancing:
            enabled: true
            type: EnvoyProxy
        podSecurityPolicy:
          defaultPolicy: 00-k0s-privileged
        storage:
          type: etcd
        telemetry:
          enabled: true
        controllerManager:
          extraArgs:
            bind-address: "0.0.0.0"
        scheduler:
          extraArgs:
            bind-address: "0.0.0.0"
DEBU[0000] Preparing phase 'Connect to hosts'
INFO[0000] ==> Running phase: Connect to hosts
DEBU[0000] [ssh] 192.168.0.2:22: using an unencrypted private key from /root/.ssh/id_rsa
DEBU[0000] [ssh] 192.168.0.2:22: executing `uname | grep -q Linux`
DEBU[0000] [ssh] 192.168.0.2:22: executing `cat /etc/os-release || cat /usr/lib/os-release`
DEBU[0000] [ssh] 192.168.0.2:22: NAME="AlmaLinux"
DEBU[0000] [ssh] 192.168.0.2:22: VERSION="9.3 (Shamrock Pampas Cat)"
DEBU[0000] [ssh] 192.168.0.2:22: ID="almalinux"
DEBU[0000] [ssh] 192.168.0.2:22: ID_LIKE="rhel centos fedora"
DEBU[0000] [ssh] 192.168.0.2:22: VERSION_ID="9.3"
DEBU[0000] [ssh] 192.168.0.2:22: PLATFORM_ID="platform:el9"
DEBU[0000] [ssh] 192.168.0.2:22: PRETTY_NAME="AlmaLinux 9.3 (Shamrock Pampas Cat)"
DEBU[0000] [ssh] 192.168.0.2:22: ANSI_COLOR="0;34"
DEBU[0000] [ssh] 192.168.0.2:22: LOGO="fedora-logo-icon"
DEBU[0000] [ssh] 192.168.0.2:22: CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
DEBU[0000] [ssh] 192.168.0.2:22: HOME_URL="https://almalinux.org/"
DEBU[0000] [ssh] 192.168.0.2:22: DOCUMENTATION_URL="https://wiki.almalinux.org/"
DEBU[0000] [ssh] 192.168.0.2:22: BUG_REPORT_URL="https://bugs.almalinux.org/"
DEBU[0000] [ssh] 192.168.0.2:22:
DEBU[0000] [ssh] 192.168.0.2:22: ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
DEBU[0000] [ssh] 192.168.0.2:22: ALMALINUX_MANTISBT_PROJECT_VERSION="9.3"
DEBU[0000] [ssh] 192.168.0.2:22: REDHAT_SUPPORT_PRODUCT="AlmaLinux"
DEBU[0000] [ssh] 192.168.0.2:22: REDHAT_SUPPORT_PRODUCT_VERSION="9.3"
DEBU[0000] [ssh] 192.168.0.2:22: executing `[ "$(id -u)" = 0 ]`
INFO[0000] [ssh] 192.168.0.2:22: connected
DEBU[0000] Preparing phase 'Detect host operating systems'
INFO[0000] ==> Running phase: Detect host operating systems
INFO[0000] [ssh] 192.168.0.2:22: is running AlmaLinux 9.3 (Shamrock Pampas Cat)
DEBU[0000] Preparing phase 'Get admin kubeconfig'
INFO[0000] ==> Running phase: Get admin kubeconfig
DEBU[0000] [ssh] 192.168.0.2:22: executing `/usr/local/bin/k0s kubeconfig admin --data-dir=/var/lib/k0s`
DEBU[0000] Preparing phase 'Disconnect from hosts'
INFO[0000] ==> Running phase: Disconnect from hosts
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: 

...snip...

    server: https://192.168.0.2:6443
  name: k0s-cluster
contexts:
- context:
    cluster: k0s-cluster
    user: admin
  name: k0s-cluster
current-context: k0s-cluster
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: 

...snip...

   client-key-data: 

...snip...

[kke]

The first choice is the spec.api.externalAddress, if there's none then it will try to look for spec.network.controlPlaneLoadBalancing.Keepalived.virtualServers[0].ipAddress and then finally the first controller's public address.

It's possible the lookup for virtual server address is flawed, I'll look into it.

[chattytak]

I have confirmed that the latest version outputs CPLB IP. Thanks.

[chattytak]

The first choice is the spec.api.externalAddress, if there's none then it will try to look for spec.network.controlPlaneLoadBalancing.Keepalived.virtualServers[0].ipAddress and then finally the first controller's public address.

By the way, I think there are cases where you only want redundancy of the kube API server among the CPLB settings.
(when load balancing is not needed).

In that case, the parameter spec.network.controlPlaneLoadBalancing.Keepalived.virtualServers[0].ipAddress is not needed,
Only the spec.network.controlPlaneLoadBalancing.Keepalived.vrrpInstances[0].virtualIPs parameter may be set.
(Only VRRP is used, not LVS.)

Is it appropriate that k0sctl only searches for IP addresses from parameters on the virtualServers side?
I think it would be better to search for IP addresses from the vrrpInstances side as well.

[kke]

The vrrpInstances is a list of CIDRs though, I guess it could cut out the netmask, but is it then always a real address? In any case, I think the list would usually contain private addresses so maybe it's not essential for generating a kubeconfig for external kubectl use 🤔