feat: change default blockSize for calico

[cyclinder]

Signed-off-by: cyclinder [email protected]

What type of PR is this?

/kind feature

What this PR does / why we need it:

According to calico official recommendation, Default value of calico_blocksize_ipv4 is 26, and calico_blocksize_ipv6 is 122.

The current default value of calico_blocksize_ipv4 in kubespray is 24, in the case of cluster pod cidr is 18-bit mask, which means there are at most 64 blocks and each block has 256 addresses, which also equates to a maximum of 64 nodes in the cluster, otherwise there will be some problems, refer to projectcalico/calico#6160
Also, by default, each node has a maximum of 110 pod(kubelet_max_pods), and cannot use up all 256 addresses.

So we should adjust the default value of calico_block_size to 26, so that with 18-bit mask, there will be at most 2^(26-18)=256 blocks with 64 addresses per block. When a node has more than 64 pods, calico assigns 2 blocks to it (calico assigns one or more blocks to each node)

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Adjust the default value of calico blockSize ipv4 to 26, and ipv6 to 122.

[cristicalin]

Hi @cyclinder, good idea but it looks like you might have broken some assumption in the CI tests, please address the CI job failures so the PR can be accepted.

[cyclinder]

Hi @cyclinder, good idea but it looks like you might have broken some assumption in the CI tests, please address the CI job failures so the PR can be accepted.

Thanks! Let me see what's wrong.

[cyclinder]

@cristicalin I found a ci bug:

kubespray/tests/testcases/040_check-network-adv.yml

Lines 2 to 10 in 9ca5632

	- hosts: kube_node
	tasks:
	- name: Test tunl0 routes
	shell: "set -o pipefail && ! /sbin/ip ro | grep '/26 via' | grep -v tunl0"
	args:
	executable: /bin/bash
	when:
	- (ipip|default(true) or cloud_provider is defined)
	- kube_network_plugin|default('calico') == 'calico'

There are two changes needed here:

• change grep '/26 via' to grep '/ {{ calico_pool_blocksize}} | default(26)'
• change ipip|default(true) to calico_ipip_mode != Never

The reason CI kept works before was that the default calico_pool_blocksize was 24, so grep '/26 via' couldn't get anything.

If you think so, I'd fix it.

[cyclinder]

https://github.com/kubernetes-sigs/kubespray/pull/9055/checks?check_run_id=7217609171

I don't know why the cluster created by this CI job still uses the 24-bit calico_pool_blockSize, which causes calico to fail when reapplying the ippool, since the current default blockSize is 26.

I checked the job for packet_debian11-calico, which already uses a blocksize of 26 bits.

[cyclinder]

Hi @cristicalin , All the tests passed, PTAL :)

[oomichi]

Nice work!

/lgtm

[liupeng0518]

Is it appropriate to modify the existing env directly when the kubespray version is upgraded?

kubelet_max_pods: 250

[cyclinder]

I think it's appropriate in most scenarios, but there may be some variables that can't be updated, such as calico_pool_blocksize

[cristicalin]

Thanks @cyclinder for this work!

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cristicalin, cyclinder

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cristicalin]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rptaylor]

@cyclinder @cristicalin It looks like this causes upgrades to kubespray 2.20 to fail. When running the kubespray/upgrade-cluster.yml playbook:

TASK [network_plugin/calico : Check if inventory match current cluster configuration] ***************************************************************************************
fatal: [cluster-dev-k8s-master-nf-1]: FAILED! => {
    "assertion": "calico_pool_conf.spec.blockSize|int == (calico_pool_blocksize | default(kube_network_node_prefix) | int)",
    "changed": false,
    "evaluated_to": false,
    "msg": "Your inventory doesn't match the current cluster configuration"
}

For any cluster built with the default config before this change, calico_pool_conf.spec.blockSize is 24. But with this PR changing calico_pool_blocksize to 26 it causes the assertion to fail here: https://github.com/kubernetes-sigs/kubespray/blob/release-2.20/roles/network_plugin/calico/tasks/check.yml#L158

How are clusters meant to be upgraded to Kubespray 2.20.0? Is there a bug fix for this that can be cherrypicked ?

Or is there a manual procedure that is required to update the block size of the calico pool? There is nothing in the 2.20 release notes about this.

[rptaylor]

Looks like #10516 could fix it but that isn't even in 2.24 yet.
I guess existing clusters need to keep the same value of calico_pool_blocksize in their inventory to avoid breakage from the change of default value?

[cyclinder]

I guess existing clusters need to keep the same value of calico_pool_blocksize in their inventory to avoid breakage from the change of default value?

Yes, you should configure the value of calico_pool_blocksize when you upgrade your cluster. Otherwise, calico_pool_conf.spec.blockSize doesn't match the calico_pool_blocksize. We can consider cherrypick #10516.