Skip to content

Releases: Azure/AKS

Release 2024-04-11

23 Apr 12:09
@aritraghosh aritraghosh
2024-04-11
c2a17e4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-04-11

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Support upgrade version skew policy between core node and control plane components from n-2 to n-3 to match related upstream policy change starting Kubernetes version 1.28. AKS docs available here.
  • Starting 1.30 Kubernetes version and 1.27 LTS versions, beta apis will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta apis closer to the 1.30 release.
  • On 15 March 2027, Windows Server 2022 will be retired when Kubernetes 1.34 reaches the end of platform support. You won't be able to create new Windows Server 2022 node pools on Kubernetes 1.35 and above. We encourage you to make the switch before 15 March 2027 to gain the richer benefits of Windows Server 2025 or Windows Server Annual Channel. These new Windows OS versions will be supported on AKS before Windows Server 2022 is retired. For more updates, see our AKS public roadmap.
  • Kubernetes version 1.26 is now removed. Refer to for platform support timeline.
  • In 2020 Docker enacted a Rate Limiting policy for all users. In-order to assist customers with the change, Microsoft worked directly with Docker to prevent users of Microsoft Azure from being impacted. However, beginning on June 30th, 2024, Azure customers will begin to be impacted by this limit. In-order for customers to mitigate the potential effects of this limit. We recommend customers begin to use the Artifact Cache feature within Azure Container Registry or sign up for a Docker Subscription. More information is available here

Release Notes

  • Features:

  • Behavioral Changes:

    • This introduces the constraint template validation behavior change called out in November's release notes
      2023-11-28 . It also improves cleanup of the addon, as called out in Issue #3541 , and patches CVE-2024-24786 in the addon.
    • Added resource nodes/proxy to microsoft-defender-operator role
    • AKS will be fixing a behavior where manually added Labels, Taints and Annotations are incorrectly copied to surged upgrade nodes. To ensure any Label or Taint is present in new nodes please use the Labels and/or Taints functionality provided by AKS.

  • Bug Fixes:

    • Fixes a bug where a PUT operation(Update) on nodepool without a specified version in LTS clusters would have an internal error.
    • Error message improved to specify that it is only allowed to update public SSH key in preview API versions.
    • Clusters running Kubernetes 1.29 or later will have kubernetes.azure.com/managedby=aks label to tigera-operator deployment in Calico clusters

  • Component Updates:

Assets 2
Loading

Release 2024-03-31

11 Apr 21:30
@abubinski abubinski
2024-03-31
12538d9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-03-31

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Support upgrade version skew policy between core node and control plane components from n-2 to n-3 to match related upstream policy change starting Kubernetes version 1.28 AKS docs available here.
  • Starting 1.30 Kubernetes version and 1.27 LTS versions, beta apis will be disabled by default, when you upgrade to them. There will be an option provided to explicitly enable beta apis closer to the 1.30 release.
  • On 15 March 2027, Windows Server 2022 will be retired when Kubernetes 1.34 reaches the end of platform support. You won't be able to create new Windows Server 2022 node pools on Kubernetes 1.35 and above. We encourage you to make the switch before 15 March 2027 to gain the richer benefits of Windows Server 2025 or Windows Server Annual Channel. These new Windows OS versions will be supported on AKS before Windows Server 2022 is retired. For more updates, see our AKS public roadmap.

Release Notes

  • Features:

    • AKS Cost Analysis is now generally available. View the aggregated costs for all your AKS clusters and namespaces in a subscription and drill into infrastructure and namespaces costs of a cluster directly in Azure Portal.
    • Trusted Access on AKS cluster is generally available now.

  • Preview Features:

    • Disable SSH is in preview now. Users can disable/enable the SSH access on nodepool level.
    • Calico can now be disabled for an AKS cluster through the update operation. More info here.

  • Behavioral Changes:

    • Customizations to HorizontalPodAutoscaler (HPA) for istiod and Istio ingress gateways are now allowed. User can directly edit the HPAs in aks-istio-system and aks-istio-ingress namespaces to customize the HPA. Note that HPA changes that violate minReplicas specified in the existing PDB will be rejected/reset.

  • Bug Fixes:

    • Fixed missing CalicoBlockSize when uninstalling Calico. This fixes a bug that can cause the disablement of Calico Network Policies to fail.
    • Fixed an issue where node image upgrade or nodepool deletion might result in node auto provisioning to stop provisioning new nodes.
    • Fixed bug where the RP would sometimes normalize the case of networkProfile.loadBalancerSku from the case the user input, such as 'standard' to 'Standard', which may have caused diffs in Terraform state files or other client tools that perform diffs.

  • Component Updates:

Assets 2
Loading

Release 2024-03-17

25 Mar 16:59
@alvinli222 alvinli222
2024-03-17
bfdc6ee
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-03-17

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Starting in March, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy addon will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes is cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • The ContainerService's ListOrchestratorProfiles API has been deprecated. Please use the ManagedCluster's ListKubernetesVersion API.
  • Changes to kube-reserved memory reservations are now in effect in AKS 1.29. The optimized reservation logic reduces kube-reserved memory by up to 20% depending on the node configuration. For existing 1.29 node pools created prior to 2/26, please perform a node pool update or recreate to see these changes. Learn more.
  • On 15 March 2027, Windows Server 2022 will be retired when Kubernetes 1.34 reaches the end of platform support. You won't be able to create new Windows Server 2022 node pools on Kubernetes 1.35 and above. We encourage you to make the switch before 15 March 2027 to gain the richer benefits of Windows Server 2025 or Windows Server Annual Channel. These new Windows OS versions will be supported on AKS before Windows Server 2022 is retired. For more updates, see our AKS public roadmap.

Release notes

  • Features

    • Kubernetes 1.29 is GA.
    • 5,000 Node Limit by Default is generally available in AKS. This limit is available for Standard tier and Premium tier clusters. The rollout for this feature will be separate from the 3/17 release. Please follow this GitHub issue for the most up to date regions where this feature has been rolled out.
    • Gen 2 VMs are now generally available for Windows on AKS. Azure Generation 2 (Gen2) virtual machines (VMs) support key features not supported in generation 1 VMs (Gen1).
    • Custom kubelet configuration is now generally available for Windows on AKS. To request additional kubelet parameters supported by Windows, create a feature request on AKS Github Issues.
    • Outbound type migration is now generally available on AKS. You can migrate egress outbound types on existing clusters without having to recreate a cluster.

  • Preview features

  • Behavioral change

    • Workload Identity is now supported as a setting for static PVs on Managed Blob/File CSI drivers in 1.29.
    • Starting with the 2024-03-01 api, OSType will reject unknown inputs.

  • Bug fixes

    • Fixed a bug where clusters with legacy hard taints on system pools could not run any operations.
    • Fixed a bug where node taints may be overwritten on certain PUT requests.
    • Fixed a bug where clusters running LTS could get a list of non-LTS versions to upgrade to.
    • Fixed a bug with Application Gateway Ingress Controller where it is unable to fetch secret objects during cluster upgrade.

  • Component updates

Assets 2
Loading

Release 2024-02-26

01 Mar 17:06
@shashankbarsin shashankbarsin
2024-02-26
0e3411c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-02-26

Release 2024-02-26

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Starting in March, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy addon will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes is cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.

Release notes

  • Features

  • Preview features

  • Behavioral change

    • ignoreUnfixed is now set to false in scanner options for Image Cleaner so that images with vulnerabilities are deleted even if there is no fix/patch available for it yet.
    • Label kubernetes.azure.com/managedby: aks has been introduced to all managed addon components on cluster. Related issue can be found here

  • Bug fixes

    • Pod overhead of memory 2Gi added to kata-cc-isolation RuntimeClass to address issue where too many pods being created to use too much of the node's memory was resulting in random processes being OOM killed.
    • Fixed issue that was causing PUT operations on AKS clusters that were using Bring your own Container Network Interface (CNI) plugin to fail when the request didn't contain the networkProfile.podCIDR property.
    • In AKS clusters of version >= 1.27.0, fixed a race condition in the iptables mode of kube-proxy that could result in some updates getting lost (for example, when a service gets a new endpoint).
    • Fixed a race condition that could cause upgrade from kubenet to Azure CNI Overlay to fail.

  • Component updates

    • Istio revision asm-1-20 is now available with Istio-based service mesh add-on. More information on performing canary upgrade for the new minor revision of Istio can be found here. Istio revision asm-1-18 is no longer supported.
    • Open Service Mesh upgraded to v1.2.8 with Envoy upgraded to v1.26.7 to address vulnerabilities CVE-2024-23324, CVE-2024-23325, CVE-2024-23322, CVE-2024-23323, and CVE-2024-23327.
    • For Node Auto Provisioning, Karpenter is upgraded to v0.33.0 and its Azure provider is upgraded to v0.3.0.
    • Upgraded Azure Disk CSI driver version to v1.26.9 on AKS 1.26, v1.28.6 on AKS 1.27, v1.29.3 on AKS 1.28.
    • Upgraded Azure File CSI driver version to v1.26.11 on AKS 1.26, v1.28.8 on AKS 1.27, v1.29.3 on AKS 1.28.
    • Upgraded Azure Blob CSI driver version to v1.21.7 on AKS 1.26, v1.22.5 on AKS 1.27, v1.23.3 on AKS 1.28.
    • Upgraded kappie-agent Linux and Windows images used in AKS Network Observability to v0.1.4 and v0.1.3 respectively.
    • Upgraded ACI provider for the Virtual Kubelet to v1.6.1
    • Cilium version has been updated to 1.14.4 for AKS clusters with kubernetes versions >= 1.29.0.
    • Azure Linux image has been updated to Azure Linux - 202402.12.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202402.12.0.
    • Windows Server 2019 Image has been updated to Windows Server 2019 - 17763.5458.240218.
    • Windows Server 2022 Image has been updated to Windows Server 2022 - 20348.2322.240218.
Assets 2
Loading

Release 2024-02-07

20 Feb 19:18
@wangyira wangyira
2024-02-07
4317e71
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-02-07

Release 2024-02-07

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Starting in March, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy addon will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • All current AKS API versions silently ignore unknown fields. An unknown field is a field that isn't part of the AKS API. AKS API version 2024-01-01, 2024-01-02-preview and all subsequent API versions will change this behavior. Unknown fields in a request will result in the request being rejected with an error stating that the unknown field is not understood. This change only impacts new API versions and won't impact you unless you update to use an API version 2024-01-01 or later. Existing API calls (via Azure Resource Manager templates or otherwise) will continue to function as-is.

Release notes

  • Features

  • Preview features

  • Bug Fixes

    • Enable HonorPVReclaimPolicy for CSI drivers on AKS 1.27+ to align with upstream behavior.
    • Node Auto Provision can now be enabled when aadProfiles, including ServerAppID, ClientAppID, ServerAppSecret, are being set.

  • Behavioral Change

    • Update the Agentpool Profile protocol to include the new PodIPAllocationMode property.

  • Component Updates

    • Istio-based service mesh add-on's istiod and ingress images updated to 1.18.7-hotfix.20240210 and 1.19.7 for asm-1-18 and asm-1-19 respectively. User needs to restart the workload pods to trigger re-injection of the newer patch version of istio-proxy. Vulnerabilities CVE-2024-23322, CVE-2024-23323, CVE-2024-23324, CVE-2024-23325, and CVE-2024-23327 have been addressed in these patch versions. More information can be found here.
    • For the cloud-provider-node-manager-windows component, the following versions have been updated:
      • v1.29.0 for >=1.29.0 version
      • v1.28.5 for >=1.28.0 version
      • v1.27.13 for >=1.27.0 version
      • v1.26.19 for >=1.26.0 version
      • v1.25.24 for >=1.25.0 version
    • Upgraded konnectivity-agent image version from v0.0.33-hotfix.20221110 to to v0.1.6-hotfix.20240116.
    • Upgraded Cilium to v1.13.10 for kubernetes v1.28.0+.
    • Upgraded Tigera Operator to v1.30.7, azurefile-csi-driver to v1.29.3, and Microsoft Defender for Cloud Low Level Collector to v.2.0.0 starting with Kubernetes v1.29 preview.
      • Calico v3.26.3 is installed when using Tigera Operator v1.30.7.
      • Microsoft Defender for Cloud Low Level Collector v.2.0.0 includes a new process collection engine, optimized and reduced CPU & Memory usage.
    • Upgraded Network Observability (Retina) to v0.1.3 with minor bug fixes.
    • Upgraded gatekeeper to v3.14.0 and policy addon v1.3.0
      • Azure Policy Changes
        • Introduces error state for policies in error, enabling them to be distinguished from policies in noncompliant states.
        • Adds support for v1 constraint templates and use of the excludedNamespaces parameter in mutation policies.
        • Adds an error status check on constraint templates post-installation.
    • Upgraded container insights agent to v3.1.17.
    • Upgraded Microsoft Defender for Cloud Security Publisher to 1.0.78 with improved logging, fixed a small bug related to cgroupv2.
    • Azure Linux image has been updated to Azure Linux - 202402.07.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202402.07.0.
    • Azure Windows 2019 Image has been updated to Azure Windows 2019 - 17763.5329.240202.
    • Azure Windows 2022 Image has been updated to Azure Windows 2022 - 20348.2227.240202.
Assets 2
Loading

Release 2024-01-23

03 Feb 03:18
@shashankbarsin shashankbarsin
2024-01-23
f91c376
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-01-23

Release 2024-01-23

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 was deprecated on January 14, 2024 and support transitions to platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • All current AKS API versions silently ignore unknown fields. An unknown field is a field that isn't part of the AKS API. AKS API version 2024-01-01, 2024-01-02-preview and all subsequent API versions will change this behavior. Unknown fields in a request will result in the request being rejected with an error stating that the unknown field is not understood. This change only impacts new API versions and won't impact you unless you update to use an API version 2024-01-01 or later. Existing API calls (via Azure Resource Manager templates or otherwise) will continue to function as-is.

Release notes

  • Features

  • Preview features

    • Istio revision 1.19 is now available with Istio-based service mesh add-on. More information on performing canary upgrade for the new minor revision of Istio can be found here. Default revision of the Istio service mesh add-on for new clusters has been updated to 1.18. Istio 1.17 version is no longer supported.
    • Istio based service mesh addon now supports plugin CA to allow users to provide their own certificates and keys for signing workload certificates. More information can be found here.
    • When troubleshooting AKS nodes, for developers not having access to Kubernetes API but having access to node ARM API, node IP and node name information are now made available in this API. More information on accessing the nodes using the private IPs can be found here.
    • The application routing add-on can now manage multiple public and internal NGINX ingress controllers. Advanced ingress controller configuration is possible via a Custom Resource Definition (CRD).
    • AKS extension in VS Code has been updated to 1.4.1.

  • Bug Fixes

    • Fixed an issue that was previously preventing AKS Infiniband support for Standard_HB120-16rs_v3 SKU.
    • Fixed nodeAffinity in calico-node DaemonSet to prevent scheduling on virtual kubelet nodes.
    • Added appgw.ingress.azure.io api-group to ingress-appgw-cr ClusterRole to address missing api-group permissions error in Application Gateway Ingress Controller addon container.

  • Behavioral Change

    • Network observability addon updated with following:
      • increased limits for CPU (500m) and Memory (300Mi).
      • Fixed issue of networking observability agent crashing issue on Windows node pool of AKS clusters version >= 1.28.
      • Introduced a new init-kappie init container as part of kappie-agent DaemonSet.
      • api-resources nodes and namespaces added to kappie-cluster-reader ClusterRole.
    • Starting this month, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy addon will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.

  • Component Updates

Assets 2
Loading

Release 2024-01-08

12 Jan 00:13
@charleswool charleswool
2024-01-08
4298fb7
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2024-01-08

Release 2024-01-08

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • CIS Kubernetes V1.27 Benchmark is published which covers AKS 1.21.x through AKS 1.27.x.
  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support the validation for constraint template. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Starting with Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS Kubernetes minor version 1.29, which previews in January 2024. See release calendar.

Release notes

  • Preview features

  • Bug Fixes

    • PUT managedCluster operations on API versions (older than 2023-09-01) that didn't support serviceMeshProfile resulted in "invalid mode" error response to the API requests. This issue has now been fixed.
    • A wrong MCR URL for KEDA image in Air Gapped Cloud was previously used resulting in potential failures in enabling the KEDA addon. This issue has now been fixed.

  • Behavioral Change

    • Starting with the 2024-01-01 and 2024-01-02-preview APIs, we will begin to reject unknown fields in the request payloads. See #4060 for more details.
    • The memory limit for Azure Key Vault provider for Secrets Store CSI Driver is now increased from 200 Mi to 300Mi.
    • Expander flag is removed from AutoscalerProfile from 2023-11-01-preview API since it may cause confusion with existing Expander.

  • Component Updates

Assets 2
Loading

Release 2023-11-28

09 Dec 21:15
@kevinkrp93 kevinkrp93
2023-11-28
d18937f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2023-11-28

Release 2023-11-28

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • Staring with the 2024-01-01 and 2024-01-02-preview APIs, we will begin to reject unknown fields in the request payloads.
  • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS Kubernetes minor version 1.29, which previews in January 2024.

Release notes

Assets 2
Loading

Release 2023-11-05

13 Nov 17:38
@bmoore-msft bmoore-msft
2023-11-05
067d290
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2023-11-05

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated on January 14, 2024 and support will transition to our platform support policy. Please upgrade to Kubernetes version 1.26 or above.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI.
  • Windows containerd v1.7 will be the default container runtime for k8s v1.28+ on AKS Windows nodes. Windows Host Process (HPC) containers is GA in Windows containerd v1.7 and it has some breaking changes.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.

Release notes

  • Features
    • Kubernetes 1.28 is GA
    • Added kubernetes patch versions 1.25.15, 1.26.10, 1.27.7
    • KEDA addon is GA
  • Preview Features
    • Cluster network settings can be updated to enable Kubenet -> CNI Overlay migration - available in the CLI
  • Bug Fixes
    • Incorporated fix for irqbalance #275 a node image upgrade from 202310.19.2 will resolve the unbalanced IRQs
    • Under some conditions it was possible to set max_surge=0 which may interfere with upgrades. Now max_surge must be > 0. See Customize node surge upgrade for more information about the setting.
    • Fixed an issue where PUT operations on managedClusters or agentPools see long latency in the overall operation due to an internal network issue.
    • PATCH operations were allowed on managedClusters in a non-terminal provisioningState. This could cause an eTag mismatch and inconsistent results or failures. PATCH operations will now be block for managedClusters in a non-terminal provisioningState.
  • Behavioral Change
    • Changes to reduce the kube-reserved memory reservation and eviction threshold will not be available in 1.28 as previously shared due to a release issue. These optimizations will be releasing with AKS 1.29, which previews in January 2024.
  • Component Updates
    • Update the aks-app-routing-operator to version 0.0.7 which includes notable changes in version 0.0.6.
      • This update has 3 CVE fixes for the nginx ingress controller.
      • The following changes are also included:
        • The AJP protocol is no longer supported.
        • The whitelist-source-range annotation has been renamed to allowlist-source-range. Both are currently supported but it is recommended to move to the new annotation allowlist-source-range.
      • The custom-http-errors annotation now only supports errors between 400 and 599.
    • Azure Monitor Metrics November release to v.6.8.1
    • Update gatekeeper to v3.13.3 and policy addon 1.2.1
      • Azure Policy Changes
        • Introduce warn for policies, available in select upcoming built-in policy experiences
        • Show an exempt ComplianceReasonCode in the portal for exempt policies.
    • Update Azure Disk CSI driver version to v1.29.1 on AKS 1.28, to v1.28.4 on AKS 1.27, to v1.26.7 on AKS 1.26 and 1.25
    • Update Azure File CSI driver version to v1.29.1 on AKS 1.28, to v1.28.6 on AKS 1.27, to v1.26.9 on AKS 1.26 and 1.25
    • Update Azure Blob CSI driver version to v1.23.1 on AKS 1.28, to v1.22.3 on AKS 1.27, to v1.21.5 on AKS 1.26 and 1.25
    • Update cloud-controller-manager image to v1.27.11, v1.26.17, v1.25.22 (release notes)
    • Update to dropgz v0.0.15 to include azure-ipam v0.0.6
    • Azure Linux image has been updated to Azure Linux - 202311.07.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202311.07.0.
Assets 2
Loading

Release 2023-10-29

03 Nov 20:05
@kaysieyu kaysieyu
2023-10-29
f869ed2
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
Release 2023-10-29

Release 2023-10-29

Monitor the release status by regions at AKS-Release-Tracker.

Announcements

  • Kubernetes 1.25 is being deprecated at the end of January 2024 and support will transition to our platform support policy.
  • No new clusters can be created with Azure AD Integration (legacy). Existing AKS clusters with Azure Active Directory integration will keep working. All Azure AD Integration (legacy) AKS clusters will be migrated to AKS-managed Azure AD automatically starting from December 1st, 2023. We recommend updating your cluster with AKS-managed Azure AD before December 1st, 2023. This way you can manage the API server downtime during non-business hours.
  • Starting January 2024, due to Gatekeeper Upstream removing validation for constraint template contents at create/update time, the Azure Policy Add-On will now no longer support this. The Azure Policy Add-On will report ‘InvalidConstraint/Template’ compliance reason code for detected errors after constraint template admission. This change does not impact other compliance reason codes. Customers are encouraged to continue to follow best practices when updating Azure Policy for Kubernetes definitions (i.e. Gator CLI).
  • Windows containerd v1.7 will be the default container runtime for k8s v1.28+ on AKS Windows nodes. Windows Host Process (HPC) containers is GA in Windows containerd v1.7 and it has some breaking changes.
  • Starting Kubernetes 1.29, the default cgroups implementation on Azure Linux AKS nodes will be cgroupsv2. Older versions of Java, .NET and NodeJS do not support memory querying v2 memory constraints and this will lead to out of memory (OOM) issues for workloads. Please test your applications for cgroupsv2 compliance, and read the FAQ for cgroupsv2.
  • AKS sent out an advisory regarding CVE-2023-29332 on September 13, 2023, which impacts AKS agent nodes. Recommended mitigation is to upgrade AKS cluster and AKS node image. If impacted clusters are not upgraded, AKS will apply mitigation on customer's next cluster update operation including node OS updates and node rolling upgrades, which may cause workload disruption.

Release notes

  • Preview Features
  • Bug Fixes
    • Corrected issue where on tainted/dedicated system pools the Vertical Pod Autoscaler (VPA) deployment could end up on non-system pools.
    • Fix for issue where a Certificate Authority bundle mismatch could produce an update on the image version of the VPA webhook.
    • Fix for possible deadlock scenario between Container Network Service and Azure CNI where pod IPs would not release on pod delete and new pods would not get an IP.
    • Fix for Windows NPM crashes in k8s 1.28 with Containerd 1.7. Bug was a result of Windows NPM DaemonSet referencing a file that did not exist in its current directory.Containerd 1.7.
    • Fix for fleet clusters, so they will now be correctly set to NRG-Lockdown RestrictionLevel Restricted, instead of Unspecified. Additionally, fleet clusters within one of the undesired Unspecified states will be fixed on reconcile.
    • Fix to prevent conflict between Open Service Mesh and AKS Admission Enforcer.
    • Fix to improve response time and reduce long mc and agentpool operation latency.
  • Behavioral Change
  • Component Updates
    • Microsoft Defender for Cloud publisher image has been updated to 1.0.68 (now distroless)
    • Microsoft Defender for Cloud OldFileCleaner image has been updated to 1.4.68
    • Azure Linux image has been updated to Azure Linux - 202310.26.0.
    • AKS Ubuntu 22.04 image has been updated to AKSUbuntu-2204-202310.26.0.
Assets 2
Loading