Ensure that the tests runs against the latest k8s version

[camilamacedo86]

What do you want to happen?

Note that we usually test against the latest 3 versions. So, that means that we need to update it to run the tests against 1.20, 1.21,1.22

Extra Labels

No response

[varshaprasad96]

In case we are referring to e2e tests, we need to rewrite this particular test (

kubebuilder/test/e2e/v3/plugin_cluster_test.go

Line 301 in 5ca8884

By("creating a curl pod")

) to work with k8s 1.21 and above.

[rashmigottipati]

/cc @rashmigottipati

[hickeyma]

I can take a look at this if its not being worked on?

[rashmigottipati]

@hickeyma Please feel free to work on this.

[hickeyma]

Thanks @rashmigottipati

[hickeyma]

/assign

[hickeyma]

Before I start I wanted to first check that the local e2e test runs as expected. I am however getting the following error:

$ make test-e2e-local

[...]

STEP: deploying the controller-manager
running: make deploy IMG=e2e-test/controller-manager:gkst
STEP: validating that the controller-manager pod is running as expected
running: kubectl -n e2e-gkst-system get pods -l control-plane=controller-manager -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}
running: kubectl -n e2e-gkst-system get pods e2e-gkst-controller-manager-5c54dd976b-kkljl -o jsonpath={.status.phase}
running: kubectl -n e2e-gkst-system get pods -l control-plane=controller-manager -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}
running: kubectl -n e2e-gkst-system get pods e2e-gkst-controller-manager-5c54dd976b-kkljl -o jsonpath={.status.phase}
running: kubectl -n e2e-gkst-system get pods -l control-plane=controller-manager -o go-template={{ range .items }}{{ if not .metadata.deletionTimestamp }}{{ .metadata.name }}{{ "\n" }}{{ end }}{{ end }}
running: kubectl -n e2e-gkst-system get pods e2e-gkst-controller-manager-5c54dd976b-kkljl -o jsonpath={.status.phase}
panic: test timed out after 20m0s

[...]

Anyone know what's going wrong here?

[hickeyma]

Opened issue #2355 for time out error mentioned in #2306 (comment)

[hickeyma]

Before any changes, I tested e2e tests locally as follows:

• Kubernetes 1.20.7 : Success
• Kubernetes 1.21.2: Success
• Kubernetes 1.22.1: Conflicts

Investigating the conflicts.

[hickeyma]

Tested with Kubernetes 1.20.7, 1.21.2, 1.22.1 locally. The e2e test are running successfully. I have pushed PR #2371 with the changes and tests are passing on the different Kubernetes back to 1.14.10.

I will have a prow PR ready to go when this PR merges. @camilamacedo86 What Kubernetes releases do you then want to test with in the CI? 1.20, 1.21, 1.22 only?

[hickeyma]

kubernetes/test-infra#23928 pushed for the prow.

[estroz]

@camilamacedo86 can you explain why projects should only be tested against the latest 3 versions, when should strive to support older versions? Much like other software, users deploy a variety of k8s versions and don’t necessarily upgrade their clusters with each new minor release.

[camilamacedo86]

Hi @estroz @hickeyma,

We mandatory need to ensure support for the latest 3 k8s versions. That does not mean that we cannot do tests with the old versions as well at all. See: https://kubernetes.io/releases/

We should be testing with the latest 3 ones but we do not need to remove the tests with the previous versions. Unless we check that do all these tests began to impact we contribute with the project because it is taking too long. IHMO that is not the case. So, we can only, for now, add the tests for what we are missing and after if/when required decide to remove the tests with the older versions. (IMHO) WDYT? Would that make sense?