Release Proposal v2.15

[floryut]

Below is a proposal for release notes:

Announcements

We are looking for maintainers, reach out in #5432.

Deprecation / Removal

• Remove support for Fedora 31 (EOL)
• Remove support for Contiv CNI (Remove contiv related files #6964)
• Remove hyperkube support, no longer available in Kubernetes (Remove hyperkube from codebase #6965)
• Helm 2 can no longer be installed (Helm v3 only #6846)

Major changes

• Add support for Fedora 33 (Add Fedora 33 CI, remove Fedora 31 #7072) (see Notes)
• Add Kata Containers support to CRI-O runtime (Add Kata Containers support to CRI-O runtime #6830)
• Add RHEL support subscription registration (Add RHEL support subscription registration #6572)
• Add crun support (Enable crun support on CRI-O #6864)
• Add etcd tls cipher suites support (Add etcd tls cipher suites #7001)
• Add GCP terraform support (Added gcp terraform support #6974)
• Allow airgapped CRI-O installation (Allow configuring CRI-O for use in offline environments #6927)
• Harden reset to work in more cases (harden reset to work in more cases #6781)
• Disable Kubernetes Dashboard by default (Disable dashboard by default #6804) (see Notes)
• Add an option to force apiserver and respective client certificate to be regenerated without upgrading (Added option to force apiserver and respective client certificate to … #6403)
• Add a script to collect necessary container images and register the images to local registry (Add manage-offline-container-images.sh #7024)
• Major proxy rework on different playbooks (Rework proxy support #7095)

Applications

• Allow configuration of nodelabels in local_volume_provisioner (Allow configuration of nodelabels in local_volume_provisioner #6620)
• [Openstack] Add external_openstack_lbaas_provider setting for occm (Add external_openstack_lbaas_provider setting for occm #6566)
• [Openstack] Add security groups not managed by terraform ([Openstack] Add security groups not managed by terraform #6865)
• [Openstack] Do not apply floating IP's before router port is created ([Openstack] Make sure router port is created before attaching floating IPs #6887)
• [Openstack] Add cluster-name to external-openstack-cloud-controller-manager (Add cluster-name to external-openstack-cloud-controller-manager #7055)
• [Azure / AWS] Added support for dynamic tags in AWS and Azure (Added support for dynamic tags in AWS and Azure. #6752)

Container managers

• [All] Remove libseccomp install tasks (Remove libseccomp install tasks #7074)
• [Containerd] Add registry mirror support (containerd docker hub registry mirror support #6962)
• [Containerd] Ensure libseccomp is installed before starting containerd on CentOS 8 (Ensure libseccomp is installed before starting containerd on CentOS 8 #6922)
• [Containerd] Add download run once feature (download run once feature for containerd #6997)
• [Containerd] Allow root path and state path to be configured (Allow containerd root and state path to be configured #7098)
• [CRI-O] Use system default for storage driver by default (crio: use system default for storage driver by default #6637)
• [CRI-O] Ensure service is started and enabled (Ensure crio service state #6753)
• [CRI-O] Reset is now working when CRI is set to CRI-O (Make reset work for crio #6812)
• [CRI-O] Avoid extra restart after install and upgrade (crio: avoid extra restart after install and upgrade #6882)
• [CRI-O] Disable CRI-O restart by Multus (Disable CRI-O restart by Multus #6930)
• [CRI-O] Add registry mirror support (add crio registry mirror support #6977)
• [CRI-O] Allow to enable download_run_once (Download once for crio #6998)
• [Docker] Add CentOS 8 and Fedora 32 docker repository (Add centos8 docker repo #6747)

Network

• [Weave] Add iptables_backend to weave options (Add iptables_backend to weave options #6639)
• [Calico] Add support for Calico CNI host-local IPAM plugin (Add support for Calico CNI "host-local" IPAM plugin #6580)
• [Calico] Added ability to set VXLAN vni and port. defaults to calico's documented default (Added ability to set calico vxlan vni and port. defaults to calico's … #6678)
• [Calico] default to using kdd datastore (calico: default to using kdd datastore #6693)
• [Calico] Add retries to update calico-rr data in etcd through calicoctl (Add retries to update calico-rr data in etcd through calicoctl #6505)
• [Calico] Handle calico-rr nodes as workers so they get upgraded too (Handle calico-rr nodes as workers so they get upgraded too #6447)
• [Calico] Avoid POD restart during initial deploy (calico: avoid POD restart during initial deploy #6886)
• [Calico] Add serviceExternalIPs option for calico installation (Added support for calico serviceExternalIPs #6928)
• [Calico] Update files to handle multi-asn bgp peering conditions (Calico: update files to handle multi-asn bgp peering conditions. #6971)
• [Calico] Blacklist Calico's VXLAN interface from NetworkManager (Blacklist Calico's VXLAN interface from NetworkManager #7037)
• [Calico] Check if inventory settings match cluster settings (calico: check if inventory settings match cluster settings #6969)
• [Flannel] Add multi architeture support to flannel (Enable multi-CPU architecture in Flannel CNI #6166)

Other note worthy changes

• Allow pre-existing floating IPs to be specified with k8s_master_fips (allow pre-existing floating IPs to be specified with k8s_master_fips #6755)
• Set ansible_python_interpreter to python3 on debian (Set ansible_python_interpreter to python3 on debian (fix error with mitogen) #6633)
• Allow resource management of metrics-server container (Allowing resource management of metrics-server container.  #6652)
• Use "kubeadm join" to join masters to control plane (Use "kubeadm join" to join masters to control plane #6661)
• Add new variable allowing additionnal audit webhook server configuration (Add new variable allowing additionnal audit webhook server configuration #6726)
• Add leader election timeouts and durations to available parameters (add leader election timeouts and durations to available parameters #6691)
• Make sure node_ip is set if node is in etcd group (Make sure node_ip is set if node is in etcd group #6719)
• Install etcdctl to host when etcd deployment type is kubeadm (install etcdctl to host when etcd deployment type is kubeadm #6857)
• Chmod kubeconfig to avoid group-readable (Chmod kubeconfig to avoid group-readable #6800)
• Hold the docker-ce-cli upgrade in Debian (Hold the docker-ce-cli #6995)
• Removes apps tags from apps meta dependencies (Removes apps tags from apps meta dependencies #7041)
• Change owner to root for bin_dir directory (change owner to root for bin_dir directory #6814)
• Add an option to disable globally applying a proxy to etc/yum.conf (Adding option to disable globally applying a proxy to etc/yum.conf #6828)
• Set feature gates in kube-proxy ConfigMap (Set feature gates in kube-proxy ConfigMap #6851)
• Allow configuring container log limits for Kubelet (Allow configuring container log limits for Kubelet #6933)
• Remove executable bit from yaml and j2 files (Remove executable bit from some files #6894)
• Fails if kubeadm_version do not matches kubernetes version (Verify and alert when kubeadm_version is unsupported #6302)
• Disable docker-ce yum repo by default (Disable docker-ce yum repo by default / cleanups #7080)
• Improve reset with many tweak (Improve reset #7094)
• Small Proxy fixes (add svc,svc.{{ dns_domain }} to no_proxy) (Proxy small fixes #7102)
• Restore ability to set pod eviction timer (Restore ability to set pod eviction timer #7114) (see Notes)
• Add ping_access_ip variable to enable/disable ping test during preinstall. Enabled by default (Add ping_access_ip to control ping test #7020)
• Remove unnecessary condition check when updating server field in kube-proxy kubeconfig (Remove unnecessary condition check when updating server field in kube-proxy kubeconfig #7145)

Component versions:

• Kubernetes v1.19.7
• Etcd 3.4.13
• Docker 19.03
• Containerd 1.3.9
• CRI-O 1.19
• CNI-plugins v0.9.0
• Calico v3.16.5
• Cilium 1.8.6
• Flannel 0.13.0
• Kube-Router 1.1.1
• Multus 3.6
• Kube-ovn 1.5.2
• Weave 2.7.0
• CoreDNS 1.7.0
• Nodelocaldns 1.16.0
• Helm 3.3.4
• Nginx-ingress 0.41.2
• Cert-manager 1.0.4
• Kubernetes Dashboard v2.1.0

Known issues

• Ansible 2.10 is not supported and using it will results in errors (cf Require 2.9.0 <= Ansible version < 2.10.0 #7130)

Notes

• Kubernetes Dashboard deployment needs to be explicitly configured with dashboard_enabled: true
• Docker version for Fedora 33 needs to be set to 20.10 as they are the only packages available and validated
• Two new variables are used for this use case kube_apiserver_pod_eviction_not_ready_timeout_seconds and kube_apiserver_pod_eviction_unreachable_timeout_seconds

[floryut]

@EppO @Miouge1 @LuckySB @woopstar @oomichi @champtar
Do you have anything left in WIP/draft ? Otherwise could we start to target a release date ?

FYI I got some issue with Fedora 33 and Coredns (cf Slack), so maybe I'll split Fedora 31 removal (eol) and postpone Fedora 33 "support"... but otherwise I'm good on my end.

[champtar]

I just have my last proxy fix: #7095

[floryut]

Ps: I've tried to keep the release note up to date, but if you feel like I've missed something or if one of the wording is strange/erroneous, feedback is always welcome.
@hafe @holmesb @electrocucaracha your input is always welcome sorry to forgot to tag you.

[floryut]

I just have my last proxy fix: #7095

Great will take a look

[EppO]

are we ready yet to promote containerd as default container runtime?

[floryut]

are we ready yet to promote containerd as default container runtime?

In this one I'd say no (not enough time to get feedback) but for k8s 1.20 we could, I'm all in for that.

[champtar]

Need to switch to cgroup driver systemd by default also (it's in a PR already)

[electrocucaracha]

My only comment could be that there are some words are mixing cases, like CentOS 8, centos8, CRI-O, cri-o, Cri-O, VXLAN, vxlan other than that everything is good.

[floryut]

My only comment could be that there are some words are mixing cases, like CentOS 8, centos8, CRI-O, cri-o, Cri-O, VXLAN, vxlan other than that everything is good.

Thanks! Wording is also important as it's the first/only thing some users see so better be clean, I've updated.

[oomichi]

@EppO @Miouge1 @LuckySB @woopstar @oomichi @champtar
Do you have anything left in WIP/draft ? Otherwise could we start to target a release date ?

FYI I got some issue with Fedora 33 and Coredns (cf Slack), so maybe I'll split Fedora 31 removal (eol) and postpone Fedora 33 "support"... but otherwise I'm good on my end.

@floryut Thanks for doing this here.
I don't have any specific changes for next release at this time, I am good :-)

[EppO]

Need to switch to cgroup driver systemd by default also (it's in a PR already)

We should do both switches for the same release then, as it has the same impact on the user side (need to scrap /var/lib/containers and restart/reboot)

[champtar]

Also need to think of the procedure for the user already using containerd (me :) )

[electrocucaracha]

@floryut maybe we can implement other way to collect release notes for the next one. I suggest to use something like reno, @dhellmann explained it very well here

[floryut]

@floryut maybe we can implement other way to collect release notes for the next one. I suggest to use something like reno, @dhellmann explained it very well here

We could try 😃

[geneliu]

@floryut , wonder whether we have a target of the release date of v2.15? Thanks!

[floryut]

@floryut , wonder whether we have a target of the release date of v2.15? Thanks!

Getting closer to it, matter of days ;)

[electrocucaracha]

@floryut maybe we can implement other way to collect release notes for the next one. I suggest to use something like reno, @dhellmann explained it very well here

We could try 😃

I have created the PR for that, let me know what you think

[rptaylor]

@floryut Could #6755 please be included??

[floryut]

@floryut Could #6755 please be included??

Added