[feature request]improve hostNetwork mode of NodePool

[rambohe-ch]

What would you like to be added:

• background:

A filter named hostnetworkpropagation in YurtHub component is used for mutating pod.spec.HostNetwork field to true in order to adapt hostNetwork mode NodePool. All pods on nodes in the hostNetwork mode NodePool should share the Host network namespace because cni plugin and component(like flannel) have not been installed in the hostNetwork mode NodePool.

But there is a scenario that pods with hostNetwork=false which use the same ports maybe scheduled to the same node in hostNetwork mode NodePool, then the filter on the node mutate pod hostNetwork field from false to true, so ports conflict will happen and cause pods start failure. this problem is a out of expectation.

• solution:

1. It is not a good idea to mutate pod hostNetwork field to true directly, because this maybe cause pods failure. so i think that it is a good idea to add NodeAffinity to pods in order to avoid pods to be scheduled on the nodes in the hostNetwork mode NodePool.

2. Pods which don't want to be scheduled to nodes in hostNetwork mode NodePool should be specified by annotation explicitly, and the pod is annotation["apps.openyurt.io/exclude-host-network-pool"] = true

3. A webhook will be added in yurt-manager component for adding NodeAffinity to pods with annotation["apps.openyurt.io/exclude-host-network-pool"] = true. Because all nodes in hostNetwork mode Nodepool have labels[nodepool.openyurt.io/hostnetwork] = true, so the NodeAffinity will be as following:

  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: nodepool.openyurt.io/hostnetwork
            operator: NotIn
            values:
            - "true"

others
/kind feature

[rambohe-ch]

@gnunu @JameKeal @YTGhost PTAL

[huangchenzhao]

I'd like to try this.
/assign @huangchenzhao

[YTGhost]

@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true to pods with hostNetwork=false?

[huangchenzhao]

@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true to pods with hostNetwork=false?

What I'm wondering is, do all pods with hostNetwork=false need to be added annotation ["apps.openyurt.io/exclude-host-network-pool"] = true?

In that way, all the pods with hostNetwork=false can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict.

Or we just add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true mannually? but how to choose which pods should be added this annotation?

I don't figure it out yet, what do you think?

[rambohe-ch]

@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true to pods with hostNetwork=false?

@YTGhost @huangchenzhao I mean that end user should add this annotation(annotation ["apps.openyurt.io/exclude-host-network-pool"] = true) to pods with hostNetwork=false manually if they don't want to run the pods on edge node in hostNetwork mode NodePool.

[rambohe-ch]

@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true to pods with hostNetwork=false?

What I'm wondering is, do all pods with hostNetwork=false need to be added annotation ["apps.openyurt.io/exclude-host-network-pool"] = true?

In that way, all the pods with hostNetwork=false can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict.

Or we just add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true mannually? but how to choose which pods should be added this annotation?

I don't figure it out yet, what do you think?

@huangchenzhao End users should add annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually to pods if they don't want to run pods on edge nodes in hostNetwork mode nodepool.

This means if pods with hostNetwork=false has no annotation ["apps.openyurt.io/exclude-host-network-pool"] = true, these pods maybe scheduled on the edge node in hostNetwork mode NodePool, then port conflicts maybe happen and pods start failure. so end users can avoid pods start failure by adding annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually.

[huangchenzhao]

@rambohe-ch Can we automatically add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true to pods with hostNetwork=false?

What I'm wondering is, do all pods with hostNetwork=false need to be added annotation ["apps.openyurt.io/exclude-host-network-pool"] = true?
In that way, all the pods with hostNetwork=false can't be scheduled to nodes in hostNetwork mode NodePool, although ports conflict doesn't exists. Is this a scheduling limitation? We could have a wider range of scheduling scope for those pods which don't have ports conflict.
Or we just add the annotation ["apps.openyurt.io/exclude-host-network-pool"] = true mannually? but how to choose which pods should be added this annotation?
I don't figure it out yet, what do you think?

@huangchenzhao End users should add annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually to pods if they don't want to run pods on edge nodes in hostNetwork mode nodepool.

This means if pods with hostNetwork=false has no annotation ["apps.openyurt.io/exclude-host-network-pool"] = true, these pods maybe scheduled on the edge node in hostNetwork mode NodePool, then port conflicts maybe happen and pods start failure. so end users can avoid pods start failure by adding annotation ["apps.openyurt.io/exclude-host-network-pool"] = true manually.

understood, thanks for detailed explanation! working on that.

[rambohe-ch]

@huangchenzhao filter hostnetworkpropagation also need to be removed, so pods in HostNetwork mode nodepool will not be effected.