[MANOPD-77175] add ignorePreflightErrors support

[n549]

Description

During init, join, ugrade procedures kubeadm runs preflight procedure to do some preliminary checks. In case of any error kubeadm stops working. Sometimes it is necessary to ignore some preflight errors to deploy or upgrade successfully (for example, if /var/lib/etcd contains files/folders which cannot be removed).

Fixes # (issue)
MANOPD-77175

Solution

New section is added to the inventory:

services:
  kubeadm_flags:
    ignorePreflightErrors: Err1,Err2,...

Default errors included in ignorePreflightErrors are Port-6443, CoreDNSUnsupportedPlugins. Previously they were hardcoded in the corresponding commands.

Custom ignorePreflightErrors settings from cluster.yaml are added to default values from defaults.yaml and duplicates are removed.

How to apply

None

Test Cases

TestCase 1
Deploy a cluster with 3 masters and at least 1 worker. cluster.yaml doesn't contain kubeadm_flags section (default settings are used).

ER: the cluster is deployed successfully.

TestCase 2
Upgrade the cluster from TestCase 1 to the next kubernetes version.

ER: the cluster is upgraded successfully.

TestCase 3
Deploy a cluster with 3 masters and at least 1 worker. cluster.yaml contains kubeadm_flags section which includes Port-6443, CoreDNSUnsupportedPlugins and something else. For example, DirAvailable--var-lib-etcd, Swap, blah-blah-blah.

ER: the cluster is deployed successfully.

TestCase 4
Upgrade the cluster from TestCase 3 to the next kubernetes version.

ER: the cluster is upgraded successfully.

Checklist

• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• Integration CI passed
• Unit tests. If Yes list of new/changed tests with brief description
• There is no merge conflicts

Unit tests

Indicate new or changed unit tests and what they do, if any.

[alexarefev]

There is no changes in enrichment procedures. I'm not sure that the custom ignorePreflightErrors value is presented in artifacts like a cluster_finalized.yaml

[n549]

There is no changes in enrichment procedures. I'm not sure that the custom ignorePreflightErrors value is presented in artifacts like a cluster_finalized.yaml

Yes, enrichment procedure was not changed, because the resulting flag string is not being added to the cluster_finalized.yaml but is evaluated during kubeadm calls. cluster_finalized.yaml contains yaml configuration of ignorePreflightErrors, default or custom.

That is the question: is this ok to evaluate the flag string during kubeadm calls or it's better to evaluate the flag string once during the inventory enrichment, put the result into the enriched cluster.yaml as a new parameter and then take this string from the inventory during kubeadm calls?