[feature request]Support for bootstrapping yurthub component using node certificates.

[rambohe-ch]

What would you like to be added:
In addition to the current use of bootstrap tokens to bootstrap start yurthub, I'd like to support the use of node certificates to bootstrap start yurthub, so that yurthub can run on nodes as a regular pod rather than a static pod.

Why is this needed:
In some scenarios, kubelet may not need to use yurthub, but other components do, so that yurthub can use the kubelet's node certificates instead of having to apply for a new one.

others
/kind feature

[rambohe-ch]

/assign @rambohe-ch

[siredmar]

@rambohe-ch i'm trying to make yurthub work using k3s agent on the node and k3s on server side.

I think this isn't so far from working. The precondition is:
k3s server running, k3s agent joined. Then i try to add the yurthub static pod and let the k3s kubelet connect to yurthub.

First i tried to use the k3s bootstrap token, but the token format differs completely from the kubeadm token format, so the yurthub cannot ask for a valid cert. This behavior is baked in in yurthub to handle only kubeadm token formats which is <id>.<secret>. So the whole token handling needs adaption to work for the other token formats and so on.
Then i found this new option using node certificates to bootstrap yurthub.

This is my yurthub.yaml on my node i want to join using k3s

apiVersion: v1
kind: Pod
metadata:
  labels:
    k8s-app: yurt-hub
  name: yurt-hub
  namespace: kube-system
spec:
  volumes:
  - name: hub-dir
    hostPath:
      path: /var/lib/yurthub
      type: DirectoryOrCreate

  - name: agent-dir
    hostPath:
      path: /var/lib/rancher/k3s/agent/
      type: Directory

  containers:
  - name: yurt-hub
    image: openyurt/yurthub:latest
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: hub-dir
      mountPath: /var/lib/yurthub

    - mountPath: /etc/kubernetes/kubelet.conf
      subPath: kubelet.kubeconfig
      name: agent-dir

    - name: agent-dir
      mountPath: /etc/kubernetes/pki/ca.crt
      subPath: client-ca.crt

    - name: agent-dir
      mountPath: /var/lib/kubelet/pki/current-kubelet.pem
      subPath: client-kubelet.crt
    command:
    - yurthub
    - --v=2
    - --server-addr=https://192.168.1.46:6443
    - --node-name=$(NODE_NAME)
    - --bootstrap-mode=kubeletcertificate
    livenessProbe:
      httpGet:
        host: 127.0.0.1
        path: /v1/healthz
        port: 10267
      initialDelaySeconds: 300
      periodSeconds: 5
      failureThreshold: 3
    resources:
      requests:
        cpu: 150m
        memory: 150Mi
      limits:
        cpu: 2
        memory: 300Mi
    securityContext:
      capabilities:
        add: ["NET_ADMIN", "NET_RAW"]
    env:
    - name: NODE_NAME
      valueFrom:
        fieldRef:
          fieldPath: spec.nodeName
  hostNetwork: true
  priorityClassName: system-node-critical
  priority: 2000001000

However, on startup it complains

F0802 10:09:48.668542       1 start.go:69] validate options: bootstrap token and bootstrap file are empty, one of them must be set

I'm puzzled what bootstrap file should be used here and how does such a bootstrap file look like?

[rambohe-ch]

node certificates to bootstrap yurthub.

This is my yurthub.yaml on my node i want to join using k3s

@siredmar Thanks for your response, I think it is a bug of kubeletcertificate mode for bootstrapping kubelet, because bootstrap token or bootstrap file is not needed for kubeletcertificate mode.

would you like to post a pull request to fix this miss?