Releases: kubernetes/kops
v1.31.0-alpha.1
What's Changed
- build(deps): bump peter-evans/create-pull-request from 6.0.5 to 6.1.0 by @dependabot in #16630
- Update dependencies by @github-actions in #16639
- added release 1.29 in doc menu by @kundan2707 in #16641
- docs: create release notes for 1.30 by @justinsb in #16640
- Bump Github Actions jobs to latest distros by @rifelpet in #16644
- Fix 2 broken links in the docs by @qlijin in #16632
- Skip failing tests on older upgrade jobs migrated to EKS by @rifelpet in #16646
- Set the STS client's region via IMDS for AssumeRoleWithWebIdentity by @rifelpet in #16647
- chore: update golang to 1.22.5 by @justinsb in #16649
- refactor: simplify signature of AddS3Permissions function by @justinsb in #16650
- refactor: ApplyClusterCmd clearly returns results by @justinsb in #16655
- refactor: use types.NamespacedName for map of serviceAccounts by @justinsb in #16651
- refactor: create awsup.GetCloud helper method by @justinsb in #16656
- refactor: allow access to S3 bucket region by @justinsb in #16657
- Bump boskos dependency by @ameukam in #16648
- Bump cloudbuild to go 1.22.5 by @rifelpet in #16661
- Set Ubuntu 24.04 as default in k8s 1.31 by @rifelpet in #16662
- Update dependencies by @hakman in #16667
- Fix cluster-autoscaler priority expander config by @rifelpet in #16670
- build(deps): bump actions/dependency-review-action from 4.3.3 to 4.3.4 by @dependabot in #16674
- build(deps): bump actions/setup-go from 5.0.1 to 5.0.2 by @dependabot in #16675
- Stop skipping hostNetwork hostname e2e test by @rifelpet in #16676
- Use UTC for AWS token tests by @rifelpet in #16680
- Add new API field for VPC CNI's network policy agent image by @rifelpet in #16681
- Add the hubble-metrics service for cilium by @rifelpet in #16682
- Update dependencies by @github-actions in #16690
- Drop InTreePluginAWSUnregister feature gate for k8s 1.31 and above by @dims in #16698
- Update dependencies by @github-actions in #16699
- Replace deprecated flag with the suggested one. by @chrismgrayftsinc in #16700
- test: Fix rendering of cloudConfig.gceServiceAccount by @hakman in #16706
- Fix InTreePluginAWSUnregister feature gate removal in 1.31 by @rifelpet in #16708
- Update AWS EBS CSI Driver to 1.33.0 by @rifelpet in #16712
- test: Use latest K8s build for scale tests by @hakuna-matatah in #16701
- refactor: support multiple podCIDRs in the node patch by @justinsb in #16710
- Update README.md by @m8nt0 in #16631
- Update dependencies by @github-actions in #16719
- test: Utilize max-nodes-to-dump to leverage on large scale tests to reduce storage by @hakuna-matatah in #16718
- Use new staging etcd-manager image for testing by @rifelpet in #16722
- Add OWNERS file in tests/e2e/scenarios/scalability by @dims in #16725
- Avoid additional logging in kube-proxy for scale tests by @hakuna-matatah in #16726
- Use latest ubuntu jammy 22.04 to match GCE by @dims in #16729
- Add KOPS_VERSION_MARKER and cleanup some other settings by @dims in #16727
- Use latest master build from k8s by @dims in #16728
- Fixes already existing nodes behaviour in kops controller by @jauru in #16737
- feat(cluster-autoscaler) Implement emitPerNodegroupMetrics parameter by @alexandresavicki in #16693
- Fix table not format of docs by @lou-lan in #16745
- chore: fix function name in comment by @pengbanban in #16731
- gce: Add option to use startup script instead of user-data by @hakman in #16705
- Fix verify-golangci-lint by @hakman in #16746
- Update dependencies by @github-actions in #16747
- spotinst: Bump k8s cluster controller to v2 by @yehielnetapp in #16717
- Drop kops-ci references by @ameukam in #16764
- Allocate more resources to cloudbuild by @rifelpet in #16765
- nodeup: if apt-get tells us to run dpkg configure, run it by @justinsb in #16755
- tests: skip "service endpoints using hostNetwork" test across all clouds by @justinsb in #16771
- tests: skip NodePort tests with cilium by @justinsb in #16772
- tests: skip kube-proxy metric tests with cilium by @justinsb in #16770
- Fix tests skips for kube-router by @hakman in #16773
- Update deps to K8s v0.31.0 by @hakman in #16774
- aws: Fix conversion for instance-selector flags by @hakman in #16776
- Enable DinD for Github codespaces by @ameukam in #16780
- refactor: OptionsBuilder works on kops.Cluster by @justinsb in #16768
- dns: use resolved region rather than re-resolving every time by @justinsb in #16778
- versionbump: update golang to 1.22.6 by @justinsb in #16779
- refactor: Move GetCloudProvider to cluster by @justinsb in #16782
- cleanup: better error messages for kube-apiserver healthcheck manifest by @justinsb in #16783
- tests: create basic test for creating VMs on github actions by @justinsb in #16784
- refactor: give clear error message if challenge endpoint cannot be found by @justinsb in #16785
- Enforce GCP limit of 64 chars for cluster name in kubetest2 deployer by @dims in #16787
- tests: add simple s3 emulator for bare-metal testing by @justinsb in #16786
- metal: stub out functions to enable cluster creation by @justinsb in #16788
- Use keys specified in env vars for GCP and AWS by @dims in #16792
- refactor: move more nodeup script logic to NodeUpScript by @justinsb in #16793
- autogen: more kube asset hashes by @justinsb in #16790
- vfs: fix file replacement for SFTP by @justinsb in #16794
- chore: add hashes for containerd 1.7.16 by @justinsb in #16791
- Don't truncate the top level domains for GCP jobs by @rifelpet in #16796
- Stop skipping protocol tests for future Cilium versions by @rifelpet in #16797
- metal: more functions to enable
kops update cluster
by @justinsb in #16789 - metal: support
kops toolbox enroll
on a control-plane machine by @justinsb in #16798 - tests: capture logs etc as github artifacts by @justinsb in #16799
- Implement SELinuxMount feature gate for GCE by @jsafrane in #16801
- etcd manager static config by @justinsb in #16805
- tests: use latest dev etcd-manager image in bare-metal test by @justinsb in #16804
- Limit GCP subnet secondary alias names to 63 chars by @rifelpet in #16808
- chore: update aws pod identity webhook by @eliasscosta in #16803
- Fix ipv6 prefix detection with aws-sdk-go-v2 by @rifelpet in #16809
- Give each kops-controller controller unique names by @rifelpet in #16812
- Include all APIserver addresses for nodeup config by @rifelpet in #16813
- Discover a bastion load balancer and use it for dumping artifacts by @rifelpet in ...
v1.30.1
What's Changed
- Automated cherry pick of #16647: Set the STS client's region via IMDS for by @rifelpet in #16757
- Automated cherry pick of #16776: aws: Fix conversion for instance-selector flags by @rifelpet in #16777
- Automated cherry pick of #16778: dns: don't use IMDS region resolver when it previously failed by @rifelpet in #16781
- Automated cherry pick of #16803: chore: update aws pod identity webhook by @eliasscosta in #16811
- Automated cherry pick of #16809: Fix ipv6 prefix detection with aws-sdk-go-v2 by @rifelpet in #16815
- Release 1.30.1 by @hakman in #16825
Full Changelog: v1.30.0...v1.30.1
v1.30.0
Significant changes
Version updates
kOps 1.30 adds support for kubernetes 1.30, and updates of most of the built-in components.
This release also updates to the next generation AWS SDK (aws-sdk-go-v2), though we do not expect
user-visible changes from this update.
Improved support for Azure
kOps on Azure has been significantly improved in this release.
Embedded asset hashes
Hashes for many kubernetes assets have been embedded into the kOps binary. When using
a well-known asset, we no longer need to download the hash from a well-known location,
which slightly speeds up kOps cluster creation, and also creates a stronger chain-of-trust.
Breaking changes
None known.
Known Issues
None known.
Deprecations
-
Support for Kubernetes version 1.24 is deprecated and will be removed in kOps 1.30.
-
Support for Kubernetes version 1.25 is deprecated and will be removed in kOps 1.31.
Full changelist
- Update dependencies by @github-actions in #16407
- build(deps): bump peter-evans/create-pull-request from 6.0.1 to 6.0.2 by @dependabot in #16408
- build(deps): bump actions/checkout from 4.1.1 to 4.1.2 by @dependabot in #16409
- upgraded cert-manager to 1.12.9 by @idealhack in #16410
- aws/cert-manager: Tighten IAM permissions for cert-manager by @andsens in #16412
- Update golangci-lint to v1.57.1 by @hakman in #16418
- Update dependencies by @github-actions in #16419
- Migrate some packages to aws-sdk-go-v2 by @rifelpet in #16420
- build(deps): bump actions/dependency-review-action from 4.1.3 to 4.2.3 by @dependabot in #16421
- Migrate more packages to aws-sdk-go-v2 by @rifelpet in #16423
- refactor: move assets into their own package by @justinsb in #16426
- refactor: isolate asset construction out of cloudup by @justinsb in #16428
- Refactor: Move nodeUpConfigBuilder to nodemodel by @justinsb in #16429
- Migrate IMDS, command completion, and region validation to aws-sdk-go-v2 by @rifelpet in #16430
- Update dependencies by @github-actions in #16431
- chore: update testdata to stop using alpha versions by @justinsb in #16433
- Update EventBridge to aws-sdk-go-v2 by @rifelpet in #16432
- Migrate SQS and SSM to aws-sdk-go-v2 by @rifelpet in #16434
- chore: refactor MirroredAsset into assets by @justinsb in #16436
- Migrate IAM to aws-sdk-go-v2 by @rifelpet in #16435
- Migrate elbv2 to aws-sdk-go-v2 by @rifelpet in #16437
- build(deps): bump actions/dependency-review-action from 4.2.3 to 4.2.5 by @dependabot in #16442
- Migrate ELB to aws-sdk-go-v2 by @rifelpet in #16441
- Various aws-sdk-go-v2 fixes by @rifelpet in #16443
- Migrate aws-sdk-go/aws to aws-sdk-go-v2/aws by @rifelpet in #16444
- Update kube-router to v2.1.0 by @aauren in #16440
- Migrate autoscaling to aws-sdk-go-v2 by @rifelpet in #16446
- Migrate route53 to aws-sdk-go-v2 by @rifelpet in #16448
- Create assetdata by @justinsb in #16438
- Update Calico to v3.27.3 by @hakman in #16363
- Migrate node controller and identity to aws-sdk-go-v2 by @rifelpet in #16451
- Update dependencies by @github-actions in #16452
- Update images.md by @esbc-disciple in #16454
- azure: Various updates by @hakman in #16456
- Provide default region to route53 client by @rifelpet in #16450
- Update dependencies by @github-actions in #16461
- Migrate EC2 to aws-sdk-go-v2 by @rifelpet in #16460
- build(deps): bump peter-evans/create-pull-request from 6.0.2 to 6.0.3 by @dependabot in #16462
- kube-router: fix version and cluster ip range by @aauren in #16467
- nodeup: error message instead of segfault by @multi-io in #16473
- azure: Use lowercase for node names by @hakman in #16469
- Migrate usage of aws-sdk-go/aws/endpoints by @rifelpet in #16480
- build(deps): bump peter-evans/create-pull-request from 6.0.3 to 6.0.4 by @dependabot in #16482
- build(deps): bump actions/checkout from 4.1.2 to 4.1.3 by @dependabot in #16481
- Update dependencies for K8s v1.30.0 by @hakman in #16485
- chore: fix function names in comment by @racequite in #16476
- tests: add unit test for aws authenticator by @justinsb in #16487
- Fix getting crictl assets by @h3poteto in #16486
- Fix function name in comment by @cuiyourong in #16489
- Update dependencies by @github-actions in #16492
- aliyun: Remove some leftovers by @hakman in #16493
- build(deps): bump actions/dependency-review-action from 4.2.5 to 4.3.0 by @dependabot in #16500
- build(deps): bump actions/checkout from 4.1.3 to 4.1.4 by @dependabot in #16501
- build(deps): bump peter-evans/create-pull-request from 6.0.4 to 6.0.5 by @dependabot in #16499
- kube-router: bump version v2.1.0 -> v2.1.1 by @aauren in #16503
- all: fix function names by @knowmost in #16502
- chore: optimize the method for obtaining gopath by @weidongkl in #16504
- Update dependencies by @github-actions in #16507
- Support multiple service-account-issuers in apiserver by @zetaab in #16497
- chore: update fileAssets documentation by @mostafahussein in #16506
- azure: Fix instance deletion by @hakman in #16510
- aws: Update components before release by @hakman in #16513
- Support AuthenticationConfig in APIserver by @zetaab in #16514
- Update Cilium to v1.15.4 by @hakman in #16515
- Update metrics-server to v0.7.1 by @hakman in #16518
- build(deps): bump actions/setup-go from 5.0.0 to 5.0.1 by @dependabot in #16522
- cert-manager: Add option to set feature gates by @hakman in #16520
- hetzner: Update components before release by @hakman in #16523
- Migrate AWS Verifier to aws-sdk-go-v2 by @rifelpet in #16483
- chore: fix function names by @luchenhan in #16505
- dns: Update components before release by @hakman in #16525
- Update etcd to v3.5.13 by @hakman in #16527
- Update cluster-autoscaler to v1.30.0 by @hakman in #16530
- Increase QPS - reduce total test duration by @hakuna-matatah in #16532
- Fix postsubmit jobs by @rifelpet in #16534
- Update containerd to v1.7.16 by @hakman in #16531
- nodeup: Check if /run/systemd/resolve/resolv.conf exists on the node and use it by @zak905 in #16495
- Update node-problem-detector to v0.8.18 by @hakman in #16537
- Use addon versions for git repo clone in e2e scenarios by @rifelpet in #16544
- Initial support for Ubuntu 24.04 (Noble Numbat) by @hakman in #16542
- Bump channels with April 2024 releases by @hakman in #16547
- Bump golang to 1.22.3 by @justinsb in #16549
- Update gcp-csi driver patch version to 1.10.13 by @justinsb in #16551
- Update dependencies by @github-actions in #16552
- chore: Replace square/go-jose.v2 with go-jose/go-jose/v4 by @hakman in #16555
- cluster-autoscaler: Fix priority expander config by @hakman in htt...
v1.29.2
(Note that v1.29.1 was not released, due to an problem in the release process)
What's Changed (since v1.29.0)
- Upgrade node-termination-handler to 1.22.0 by @jim-barber-he in #16595
- Make ASG Warmpool depend on ASG Lifecycle hook by @jim-barber-he in #16603
- Support kube-controller-manager component by @chubchubsancho in #16608
- Update aws-iam-authenticator image by @rifelpet in #16616
- Update Go to v1.21.4 by @hakman in #16619
- Upgrade cilium to v1.15.6 by @rifelpet in #16628
- Update golang to 1.22.5 by @justinsb in #16653
- Fix cluster-autoscaler priority expander config by @rifelpet in #16672
- Bump cloudbuild to go 1.22.5 by @justinsb in #16684
- Add the hubble-metrics service for cilium by @rifelpet in #16687
- Add new API field for VPC CNI's network policy agent by @rifelpet in #16689
Full Changelog: v1.29.0...v1.29.2
v1.28.7
(Note that v1.28.6 was not released, due to a problem in the release process)
What's Changed (since v1.28.5)
- Support definition of kube-controller-manager by @chubchubsancho in #16609
- Update Calico to v3.27.3 by @rifelpet in #16613
- Update golang to 1.22.5 by @justinsb in #16652
- Create a dedicated staging bucket for kops builds by @justinsb in #16678
- Fix cluster-autoscaler priority expander config by @rifelpet in #16673
- Bump cloudbuild to go 1.22.5 by @justinsb in #16683
Full Changelog: v1.28.5...v1.28.7
v1.30.0-beta.1
What's Changed
- build(deps): bump actions/checkout from 4.1.4 to 4.1.5 by @dependabot in #16565
- Update dependencies by @github-actions in #16566
- fix KOPS_AWS_ROLE_ARN assume behaviour by @zetaab in #16568
- azure: Limit VMSS scope to specific storage account by @hakman in #16567
- tests: use version from go.mod in github actions by @justinsb in #16550
- Upgrade node-termination-handler to 1.22.0 by @jim-barber-he in #16572
- add service account issuer migration doc by @elliotdobson in #16541
- fix service account issuer migration doc nav by @elliotdobson in #16573
- docs: Update release notes to reflect release of 1.29 by @justinsb in #16574
- aws: Use EndpointResolverV2 for enabling dual-stack by @hakman in #16575
- Update dependencies by @github-actions in #16576
- build(deps): bump actions/checkout from 4.1.5 to 4.1.6 by @dependabot in #16579
- aws: Experiment with Adaptive retry mode by @hakman in #16580
- aws node handshake: support both v1 and v2 signatures, default to v1 by @justinsb in #16577
- tests: add mock for STS to better test node handshake by @justinsb in #16578
- Enabling statefulsets without pvs for load tests. by @hakuna-matatah in #16598
- [DOCS] Add docs for Debian 12 on AWS by @dannyeuu in #16600
- Make ASG Warmpool depend on ASG Lifecycle hook by @jim-barber-he in #16583
- Update dependencies by @hakman in #16607
- feat(components): permit to define kube-controller-manager and kube-scheduler resources by @chubchubsancho in #16605
- chore: Bump alpha channel with June releases by @hakman in #16611
- Update dependencies by @github-actions in #16614
- Update aws-iam-authenticator image by @rifelpet in #16615
- chore: Promote alpha channel to stable by @hakman in #16617
- chore: Update Go to v1.21.4 by @hakman in #16618
- Add support for Rocky 9 by @rifelpet in #16620
- Dont install libcgroup on Rocky 9 by @rifelpet in #16621
- build(deps): bump actions/checkout from 4.1.6 to 4.1.7 by @dependabot in #16622
- tests: Add test for dl.k8s.io kubernetes version parsing by @justinsb in #16624
- Raise errors earlier if we cannot parse the kube version by @justinsb in #16625
- Update dependencies by @github-actions in #16626
- Upgrade Cilium to v1.15.6 by @rifelpet in #16627
- Release 1.30.0-beta.1 by @justinsb in #16629
New Contributors
- @elliotdobson made their first contribution in #16541
- @dannyeuu made their first contribution in #16600
Full Changelog: v1.30.0-alpha.1...v1.30.0-beta.1
v1.30.0-alpha.1
What's Changed
- Update dependencies by @github-actions in #16407
- build(deps): bump peter-evans/create-pull-request from 6.0.1 to 6.0.2 by @dependabot in #16408
- build(deps): bump actions/checkout from 4.1.1 to 4.1.2 by @dependabot in #16409
- upgraded cert-manager to 1.12.9 by @idealhack in #16410
- aws/cert-manager: Tighten IAM permissions for cert-manager by @andsens in #16412
- Update golangci-lint to v1.57.1 by @hakman in #16418
- Update dependencies by @github-actions in #16419
- Migrate some packages to aws-sdk-go-v2 by @rifelpet in #16420
- build(deps): bump actions/dependency-review-action from 4.1.3 to 4.2.3 by @dependabot in #16421
- Migrate more packages to aws-sdk-go-v2 by @rifelpet in #16423
- refactor: move assets into their own package by @justinsb in #16426
- refactor: isolate asset construction out of cloudup by @justinsb in #16428
- Refactor: Move nodeUpConfigBuilder to nodemodel by @justinsb in #16429
- Migrate IMDS, command completion, and region validation to aws-sdk-go-v2 by @rifelpet in #16430
- Update dependencies by @github-actions in #16431
- chore: update testdata to stop using alpha versions by @justinsb in #16433
- Update EventBridge to aws-sdk-go-v2 by @rifelpet in #16432
- Migrate SQS and SSM to aws-sdk-go-v2 by @rifelpet in #16434
- chore: refactor MirroredAsset into assets by @justinsb in #16436
- Migrate IAM to aws-sdk-go-v2 by @rifelpet in #16435
- Migrate elbv2 to aws-sdk-go-v2 by @rifelpet in #16437
- build(deps): bump actions/dependency-review-action from 4.2.3 to 4.2.5 by @dependabot in #16442
- Migrate ELB to aws-sdk-go-v2 by @rifelpet in #16441
- Various aws-sdk-go-v2 fixes by @rifelpet in #16443
- Migrate aws-sdk-go/aws to aws-sdk-go-v2/aws by @rifelpet in #16444
- Update kube-router to v2.1.0 by @aauren in #16440
- Migrate autoscaling to aws-sdk-go-v2 by @rifelpet in #16446
- Migrate route53 to aws-sdk-go-v2 by @rifelpet in #16448
- Create assetdata by @justinsb in #16438
- Update Calico to v3.27.3 by @hakman in #16363
- Migrate node controller and identity to aws-sdk-go-v2 by @rifelpet in #16451
- Update dependencies by @github-actions in #16452
- Update images.md by @esbc-disciple in #16454
- azure: Various updates by @hakman in #16456
- Provide default region to route53 client by @rifelpet in #16450
- Update dependencies by @github-actions in #16461
- Migrate EC2 to aws-sdk-go-v2 by @rifelpet in #16460
- build(deps): bump peter-evans/create-pull-request from 6.0.2 to 6.0.3 by @dependabot in #16462
- kube-router: fix version and cluster ip range by @aauren in #16467
- nodeup: error message instead of segfault by @multi-io in #16473
- azure: Use lowercase for node names by @hakman in #16469
- Migrate usage of aws-sdk-go/aws/endpoints by @rifelpet in #16480
- build(deps): bump peter-evans/create-pull-request from 6.0.3 to 6.0.4 by @dependabot in #16482
- build(deps): bump actions/checkout from 4.1.2 to 4.1.3 by @dependabot in #16481
- Update dependencies for K8s v1.30.0 by @hakman in #16485
- chore: fix function names in comment by @racequite in #16476
- tests: add unit test for aws authenticator by @justinsb in #16487
- Fix getting crictl assets by @h3poteto in #16486
- Fix function name in comment by @cuiyourong in #16489
- Update dependencies by @github-actions in #16492
- aliyun: Remove some leftovers by @hakman in #16493
- build(deps): bump actions/dependency-review-action from 4.2.5 to 4.3.0 by @dependabot in #16500
- build(deps): bump actions/checkout from 4.1.3 to 4.1.4 by @dependabot in #16501
- build(deps): bump peter-evans/create-pull-request from 6.0.4 to 6.0.5 by @dependabot in #16499
- kube-router: bump version v2.1.0 -> v2.1.1 by @aauren in #16503
- all: fix function names by @knowmost in #16502
- chore: optimize the method for obtaining gopath by @weidongkl in #16504
- Update dependencies by @github-actions in #16507
- Support multiple service-account-issuers in apiserver by @zetaab in #16497
- chore: update fileAssets documentation by @mostafahussein in #16506
- azure: Fix instance deletion by @hakman in #16510
- aws: Update components before release by @hakman in #16513
- Support AuthenticationConfig in APIserver by @zetaab in #16514
- Update Cilium to v1.15.4 by @hakman in #16515
- Update metrics-server to v0.7.1 by @hakman in #16518
- build(deps): bump actions/setup-go from 5.0.0 to 5.0.1 by @dependabot in #16522
- cert-manager: Add option to set feature gates by @hakman in #16520
- hetzner: Update components before release by @hakman in #16523
- Migrate AWS Verifier to aws-sdk-go-v2 by @rifelpet in #16483
- chore: fix function names by @luchenhan in #16505
- dns: Update components before release by @hakman in #16525
- Update etcd to v3.5.13 by @hakman in #16527
- Update cluster-autoscaler to v1.30.0 by @hakman in #16530
- Increase QPS - reduce total test duration by @hakuna-matatah in #16532
- Fix postsubmit jobs by @rifelpet in #16534
- Update containerd to v1.7.16 by @hakman in #16531
- nodeup: Check if /run/systemd/resolve/resolv.conf exists on the node and use it by @zak905 in #16495
- Update node-problem-detector to v0.8.18 by @hakman in #16537
- Use addon versions for git repo clone in e2e scenarios by @rifelpet in #16544
- Initial support for Ubuntu 24.04 (Noble Numbat) by @hakman in #16542
- Bump channels with April 2024 releases by @hakman in #16547
- Bump golang to 1.22.3 by @justinsb in #16549
- Update gcp-csi driver patch version to 1.10.13 by @justinsb in #16551
- Update dependencies by @github-actions in #16552
- chore: Replace square/go-jose.v2 with go-jose/go-jose/v4 by @hakman in #16555
- cluster-autoscaler: Fix priority expander config by @hakman in #16556
- Use embedded hashes for our well-known assets by @justinsb in #16439
- Release 1.30.0-alpha.1 by @hakman in #16563
New Contributors
- @esbc-disciple made their first contribution in #16454
- @multi-io made their first contribution in #16473
- @racequite made their first contribution in #16476
- @cuiyourong made their first contribution in #16489
- @knowmost made their first contribution in #16502
- @weidongkl made their first contribution in #16504
- @luchenhan made their first contribution in #16505
Full Changelog: https://g...
v1.29.0
Significant changes
Deferred deletion / pruning phase
Some infrastructure changes are potentially disruptive to the continued
operation of the cluster. For the most disruptive operations, particularly
those that break rolling-update of the cluster, we have started to use deferred
deletion to minimize the impact. For example, on AWS we create a second NLB
during the kops update
phase when we cannot change the NLB directly.
kops update
will report that a --prune
is needed. To minimize disruption,
we recommend you perform this after a rolling-update, for example:
kops update $MYCLUSTER --yes --admin
kops rolling-update $MYCLUSTER --yes
kops update $MYCLUSTER --yes --admin --prune # NEW!
Deferred deletion is currently used to safely introduce security groups for NLBs on AWS,
and to move to an internal load balancer for kops-controller
on GCP.
Initial OpenTelemetry Support
We are starting to add (experimental) support for OpenTelemetry,
in particular Tracing support. Setting OTEL_EXPORTER_OTLP_TRACES_FILE
will write a trace file which can then be read by the traceserver program.
More information and options are described in docs/opentelemetry.md.
The tracing data is not expected to be particularly useful for end-users in
this release; the (non-standard) recording approach is instead intended to
work well with our Prow end-to-end testing system so that developers can
optimize kOps.
Please note: this is not telemetry in the "phone-home" sense.
The kOps project does not collect data from your machine. As an
open-source project we do not even want to collect any of your data.
Currently the only OpenTelemetry backend supported is writing to a
filesystem (and it is opt-in). In future you will be able to configure
other OpenTelemetry backends, but this data will only be sent if
you enable OpenTelemetry, and only sent to where you configure.
AWS
-
Network Load Balancers in front of the Kubernetes API and bastion hosts now
have a security group attached. These security groups are used for security group rules
allowing incoming traffic to the NLBs as well as traffic between the NLBs and their target
instances. -
Posts event data to URL upon instance interruption action in aws-node-termination-handler with
WEBHOOK_URL
.
GCP
-
As of Kubernetes version 1.29, credentials for private GCR/AR repositories will be handled by the out-of-tree credential provider. This is an additional binary that each instance downloads from the assets repository.
-
Two additional
StorageClasses
are created on GCP clusters. These are calledbalanced-csi
andssd-csi
and utilize the GCP Balanced and SSD Persistent Disk volume types respectively. -
Breaking Change - the default
StorageClass
has been changed fromstandard-csi
tobalanced-csi
. -
We now use a private load-balancer for in-cluster traffic on GCP, which allows us
to use network tags to restrict access only to the cluster nodes.
Breaking changes
Other breaking changes
-
kops toolbox dump
limits the number of nodes dumped to 500 by default. Use--max-nodes
to override. -
Support for Kubernetes version 1.23 has been removed.
Known Issues
- The Amazon VPC CNI is now compatible with Ubuntu 22.04. Fix applied via kubernetes/kops#16313.
Deprecations
-
Support for Kubernetes version 1.24 is deprecated and will be removed in kOps 1.30.
-
Support for Kubernetes version 1.25 is deprecated and will be removed in kOps 1.31.
-
Support for AWS Classic Load Balancer for API is deprecated and should not be used for newly created clusters.
-
All unmanaged addons (in
addons/
) are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.
What's Changed
- aws: Add option to provide multiple instance types on cluster creation by @hakman in #15854
- Release 1.29.0-alpha.0 by @hakman in #15858
- Fix warmpool to expose dependencies for dependency analysis by @justinsb in #15848
- Remove support for Kubernetes 1.23 by @johngmyers in #15860
- Only run one replica of controller pods on non-HA clusters by @justinsb in #15868
- Default to 100.64.0.0/13 as IPv4 service cluster IP range by @hakman in #15866
- Improve status printing when tasks are in progress by @justinsb in #15847
- Clarify comment on nonMasqueradeCIDR field. by @justinsb in #15869
- Use mixed instances for scale tests by @hakman in #15859
- Remove dead code for in-tree CCM by @johngmyers in #15872
- Legacy control-plane node labels are no longer used by @johngmyers in #15874
- Allow setting additional flags for apiserver, kcm and scheduler by @hakman in #15877
- Allow setting resource.Quantity from the command line by @hakman in #15876
- Tune apiserver, kcm and scheduler flags for scale performance tests by @hakuna-matatah in #15875
- Remove Docker config option by @hakman in #15871
- Update ko to v0.14.1 by @hakman in #15879
- Update Calico to v3.25.2 by @hakman in #15878
- ENI Cleanup Improvements by @rifelpet in #15884
- Release 1.29.0-alpha.1 by @johngmyers in #15889
- Update dependencies by @github-actions in #15890
- Update Go to v1.21.1 by @hakman in #15893
- Remove metadata-proxy from GCE Clusters from 1.29+ onwards by @upodroid in #15885
- Implement Cilium Ingress by @zadjadr in #15795
- Bump actions/dependency-review-action from 3.0.8 to 3.1.0 by @dependabot in #15894
- Bump actions/checkout from 3.6.0 to 4.0.0 by @dependabot in #15895
- Handle 404 properly from Service Accounts API by @upodroid in #15902
- Increase vpc cidrs to allocate more ip's for large scale k8s clusters by @hakuna-matatah in #15904
- kops-controller: Increase client-side throttling limits by @hakman in #15906
- Increase the etcd db size to accomodate large scale k8s cluster resources by @hakuna-matatah in #15908
- Add Cognito permissions for AWS LBC by @danports in #15910
- Fix COS image mounter URL by @upodroid in #15907
- scale-test: Increase validation wait time to 45m by @hakman in #15915
- Make
--cloudonly
sound less innocuous by @danports in #15911 - Update dependencies by @github-actions in #15917
- scale-test: Add few more cidrs to divide equally /16's by @hakuna-matatah in #15920
- Update sync target for OWNERS by @johngmyers in #15921
- Add
SeccompDefault
kubelet config by @colinhoglund in #15919 - Optimize subnets asignment to IGs for clusters with multiple CIDRs by @hakman in #15923
- fix a few log with percent w by @justinsb in #15926
- Revert "WIP: Need to publish IPv4 DNS records" by @justinsb in #15928
- fix some comments by @justinsb in #15927
- scale-test: Need for separate etcd cluster for events by @hakuna-matatah in #15932
- scale-test: Optimise node dumping for large clusters by @hakman in #15934
- scale-test: Re-enable etcd metrics by @hakman in #15938
- protokube: support writing AAAA records to /etc/hosts by @justinsb in #15931
- Update dependencies by @github-actions in #15944
- Cilium: Implement node encryption by @zadjadr in #15901
- Fix e2e testing logic by @upodroid in #15900
- aws: Fix minor typos for Karpenter setup by @junm-cloudnatix in #15935
- aws: Configure Karpenter resources in the cluster spec by @danports in #15916
- Update 1.28 release notes for release by @johngmyers in #15956
- gce ipv6: we can use network tags with ipv6 addresses by @justinsb in #15955
- Bump actions/checkout from 4.0.0 to 4.1.0 by @dependabot in #15960
- Update Karpenter to v0.30.0 by @hakman in #15945
- aws: Filter ENI list by VPC by @hakman in #15903
...
v1.28.5
What's Changed
- fix(nodeup): set
MACAddressPolicy
tonone
when using AWS by @hakman in #16319 - Added image minimum and maximum gc age by @hakman in #16320
- validation: Allow overlap of pod/node CIDR and service CIDR by @hakman in #16345
- Bump metrics-server to 0.6.4 by @idealhack in #16398
- Bump cert manager to 1.12.7 by @idealhack in #16399
- aws: expose port 8443 when using NLB with a custom by @hakman in #16405
- Upgraded cert-manager to 1.12.9 by @idealhack in #16411
- azure: Use lowercase for node names by @hakman in #16471
- Support multiple service-account-issuer flags by @hakman in #16512
- aws: Update CCM to v1.28.5 by @hakman in #16524
- Update etcd to v3.5.13 by @hakman in #16529
- Update containerd to v1.7.16 by @hakman in #16536
- check /run/systemd/resolve/resolv.conf inside by @hakman in #16539
- Update dependencies before release for v1.28 by @hakman in #16554
- cluster-autoscaler: Fix priority expander config by @hakman in #16559
Full Changelog: v1.28.4...v1.28.5
v1.29.0-beta.1
This is the first beta of the 1.29 release.
Significant changes
Deferred deletion / pruning phase
Some infrastructure changes are potentially disruptive to the continued
operation of the cluster. For the most disruptive operations, particularly
those that break rolling-update of the cluster, we have started to use deferred
deletion to minimize the impact. For example, on AWS we create a second NLB
during the kops update
phase when we cannot change the NLB directly.
kops update
will report that a --prune
is needed. To minimize disruption,
we recommend you perform this after a rolling-update, for example:
kops update $MYCLUSTER --yes --admin
kops rolling-update $MYCLUSTER --yes
kops update $MYCLUSTER --yes --admin --prune # NEW!
Deferred deletion is currently used to safely introduce security groups for NLBs on AWS,
and to move to an internal load balancer for kops-controller
on GCP.
Initial OpenTelemetry Support
We are starting to add (experimental) support for OpenTelemetry,
in particular Tracing support. Setting OTEL_EXPORTER_OTLP_TRACES_FILE
will write a trace file which can then be read by the traceserver program.
More information and options are described in docs/opentelemetry.md.
The tracing data is not expected to be particularly useful for end-users in
this release; the (non-standard) recording approach is instead intended to
work well with our Prow end-to-end testing system so that developers can
optimize kOps.
Please note: this is not telemetry in the "phone-home" sense.
The kOps project does not collect data from your machine. As an
open-source project we do not even want to collect any of your data.
Currently the only OpenTelemetry backend supported is writing to a
filesystem (and it is opt-in). In future you will be able to configure
other OpenTelemetry backends, but this data will only be sent if
you enable OpenTelemetry, and only sent to where you configure.
AWS
-
Network Load Balancers in front of the Kubernetes API and bastion hosts now
have a security group attached. These security groups are used for security group rules
allowing incoming traffic to the NLBs as well as traffic between the NLBs and their target
instances. -
Posts event data to URL upon instance interruption action in aws-node-termination-handler with
WEBHOOK_URL
.
GCP
-
As of Kubernetes version 1.29, credentials for private GCR/AR repositories will be handled by the out-of-tree credential provider. This is an additional binary that each instance downloads from the assets repository.
-
We now use a private load-balancer for in-cluster traffic on GCP, which allows us
to use network tags to restrict access only to the cluster nodes.
Breaking changes
-
kops toolbox dump
limits the number of nodes dumped to 500 by default. Use--max-nodes
to override. -
Support for Kubernetes version 1.23 has been removed.
Known Issues
- The Amazon VPC CNI is now compatible with Ubuntu 22.04. Fix applied via kubernetes/kops#16313.
Deprecations
-
Support for Kubernetes version 1.24 is deprecated and will be removed in kOps 1.30.
-
Support for Kubernetes version 1.25 is deprecated and will be removed in kOps 1.31.
-
Support for AWS Classic Load Balancer for API is deprecated and should not be used for newly created clusters.
-
All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.
What's Changed
kops validate cluster
improvements by @upodroid in #16187- gce: Remove custom resolver by @hakman in #16189
- skip_regex.go: kube-router add back in service afinity test by @aauren in #16188
- chore(channels): bump k8s and ubuntu ami versions in alpha channel by @moshevayner in #16190
- Update Calico to v3.27.0 by @hakman in #16192
- Disable Statefulsets provisioning from CL2 Load Tests by @hakuna-matatah in #16172
- Add cert-manager resource types to kubetest2-kops artifacts by @rifelpet in #16193
- Parallelize k8s resource dumps with kops toolbox dump by @rifelpet in #16196
- Include pod logs in toolbox dump by @rifelpet in #16198
- Update k8s.io/* to v0.29.0 by @hakman in #16199
- Update dependencies by @github-actions in #16201
- test: Print the
create cluster
command by @hakman in #16202 - scale-test: Add feature flag for creating a single nodes instance group by @hakman in #16203
- Dump previous and current container logs separately by @hakman in #16200
- aws: Set AWS_REGION env var for ebs-csi-node and ebs-csi-driver by @hakman in #16206
- aws: Add option for setting QPS and Burst for EBS CSI Driver by @hakman in #16207
- Spotinst: Bump controller version to 1.0.97 by @yehielnetapp in #16208
- feat: add us-west zone for hetzner by @finzzz in #16209
- Check if kubeconfig exists before dumping resources by @hakman in #16205
- Promote alpha to stable by @moshevayner in #16210
- aws: Use instance metadata to get warm pool state by @rifelpet in #16213
- Dump and redact secrets by @rifelpet in #16211
- Update to Cilium 1.14.5 by @hakman in #16214
- Allow override of the DNS domain used by the tests. by @ameukam in #16217
- aws: Retrieve instance info only when max pods is not set by @hakman in #16216
- Add permission needed for service-linked role creation by @ameukam in #16219
- Remove kube-system cert-manager webhook exclusion by @rifelpet in #16221
- Jaeger tracing visualizer improvements by @rifelpet in #16220
- Update dependencies by @github-actions in #16224
- aws: Set provider ID when starting kubelet by @hakman in #16223
- scale-test: Reduce validation count and interval by @hakman in #16225
- aws: Update EBS CSI driver to v1.26.0 by @hakman in #16227
- Add option for setting CCM ConcurrentNodeSyncs by @hakman in #16228
- aws: Skip deleting ASG instances without volumes by @hakman in #16229
- Make cluster deletion configurable by @hakman in #16231
- Bump actions/dependency-review-action from 3.1.4 to 3.1.5 by @dependabot in #16232
- scale-test: Use single nodes instance group for AWS by @hakman in #16204
- Bump kubetest2 by @ameukam in #16234
- test: Improve cluster deletion defaults by @hakman in #16236
- Replace
k8s.io/utils/strings/slices
withgolang.org/x/exp/slices
by @hakman in #16238 - aws: Update EBS CSI driver by @hakman in #16239
- aws: Use
domain
instead ofvpc
when renderingaws_eip
by @hakman in #16237 - Bump GCP terraform provider to latest by @rifelpet in #16242
- docs: fix broken example command by @markusleh in #16243
- Update dependencies by @github-actions in #16244
- Revert "aws: Skip deleting ASG instances without volumes" by @hakman in #16246
- Update Go to v1.21.6 by @hakman in #16245
- Prefer external endpoints when building kubeconfig by @justinsb in #16248
- aws: Terminate ASG instances in batches of 100 instances by @hakman in #16251
- aws: Ignore InvalidRouteTableID.NotFound errors during cluster deletion by @hakman in #16252
- aws: fix maxPods when cilium ipam=eni is used by @argusua in #16253
- Update containerd to v1.7.12 by @hakman in #16257
- Switch to GCS url for upgrades tests by @ameukam in #16258
- Use dns=none for newly created clusters including for AWS and GCE by @hakman in #16262
- Update aws-sdk-go to v1.49.24 by @ameukam in #16263
- test: Set num-nodes flag by @upodroid in #16176
- Refactor: Replace ForAPIServer with WellKnownServices by @justinsb in #15829
- gce: fix nlb firewall rules, operations an...