feat: enhanced in-place update module to support vertical scaling

[LavenderQAQ]

Ⅰ. Describe what this PR does

Enhanced the in-place update module to enable kruise's workload to modify resources without restarting the pod.

Ⅱ. Does this pull request fix one issue?

Fixes #1212

Ⅲ. Describe how to verify it

Need to kubernetes v1.27 version, and open feature gate InPlacePodVerticalScaling. Here is my kind configuration:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
  "InPlacePodVerticalScaling": true
nodes:
- role: control-plane
  image: kindest/node:v1.27.3@sha256:3966ac761ae0136263ffdb6cfd4db23ef8a83cba8a463690e98317add2c9ba72
- role: worker
  image: kindest/node:v1.27.3@sha256:3966ac761ae0136263ffdb6cfd4db23ef8a83cba8a463690e98317add2c9ba72
- role: worker
  image: kindest/node:v1.27.3@sha256:3966ac761ae0136263ffdb6cfd4db23ef8a83cba8a463690e98317add2c9ba72
- role: worker
  image: kindest/node:v1.27.3@sha256:3966ac761ae0136263ffdb6cfd4db23ef8a83cba8a463690e98317add2c9ba72

Build a cloneset (or other workload that includes in-place updates) and set updateStrategy to InPlaceIfPossible. Here's an example:

apiVersion: apps.kruise.io/v1alpha1
kind: CloneSet
metadata:
  name: nginx-cloneset
spec:
  updateStrategy:
    type: InPlaceIfPossible
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx-1
        image: nginx:1.21.1
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: "1"
            memory: "256Mi"
          requests:
            cpu: "0.5"
            memory: "128Mi"

Change the workload's resources, wait a while, and you'll find that the pod's resources will change, and the pod won't restart.

Ⅳ. Special notes for reviews

The control of Status for vertical scaling requires a higher version of the k8s api, which makes this pr need to wait for kruise version upgrade. It cannot be merged at this time because it is not yet complete.

[kruise-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign veophi for approval by writing /assign @veophi in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sonatype-lift]

Sonatype Lift is retiring

Sonatype Lift will be retiring on Sep 12, 2023, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
We are extremely grateful and thank you for your support over the years.

📖 Read about the impacts and timeline

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 72.50000% with 55 lines in your changes missing coverage. Please review.

Project coverage is 49.40%. Comparing base (0d0031a) to head (eea019c).
Report is 85 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/util/inplaceupdate/inplace_update_defaults.go	77.00%	17 Missing and 6 partials ⚠️
pkg/util/inplaceupdate/inplace_update_vertical.go	76.74%	11 Missing and 9 partials ⚠️
pkg/controller/cloneset/core/cloneset_core.go	0.00%	6 Missing ⚠️
...kg/controller/cloneset/sync/cloneset_sync_utils.go	0.00%	3 Missing ⚠️
pkg/util/inplaceupdate/inplace_update.go	40.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1353      +/-   ##
==========================================
+ Coverage   47.91%   49.40%   +1.49%     
==========================================
  Files         162      192      +30     
  Lines       23491    19752    -3739     
==========================================
- Hits        11256     9759    -1497     
+ Misses      11014     8730    -2284     
- Partials     1221     1263      +42     

Flag	Coverage Δ
unittests	49.40% <72.50%> (+1.49%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[LavenderQAQ]

The complete logic needs to wait until the k8s api of kruise is upgraded.

[LavenderQAQ]

/hold

[LavenderQAQ]

I rebased it first, and I need to take some time to sort through the logic and fill in some tests. After all, this pr is a bit old.

[ABNER-1]

Some methods in VerticalUpdateInterface appear to be unimplemented.

[ABNER-1]

I encountered two errors while adding e2e test cases:

1. When modifying the image and increasing resources simultaneously, I encountered a RunContainerError. I have already submitted an issue to the Kubernetes community at [InPlacePodVerticalScaling]Got RunContainerError when patch pod image and resources kubernetes/kubernetes#127356.

2. When I scaled the CPU limit of resources from 100m to 200m, I observed that occasionally the status was first changed by kubelet from 100m InProcess to 200m, and then it was set back to 100m InProcess by kubelet, and after a while, it finally became 200m.