Improve adding controller nodes to e2e/local-testing setups

[invidian]

There are at least 2 things, which could be improved:

• Controller and worker IP addresses are right next to each other, so when having 1 controller and 1 worker, then adding additional controller, it's IP collides with the worker. Perhaps we should apply an offset to worker IPs by +5 or something, so max 5 controllers will be supported, which should be sufficient.
• promoting standalone kube-apiserver to kube-apiserver with api-loadbalancer results in ports collision at the moment. Given that HAProxy footprint is really tiny, we should just always deploy it, so we are HA-ready.

[invidian]

I also got:

Error: Provider produced inconsistent final plan

When expanding the plan for flexkube_controlplane.bootstrap to include new
values learned so far during apply, provider
"registry.terraform.io/-/flexkube" produced an invalid new value for
.state[0].container[0].config[0].args[1]: was
cty.StringVal("--etcd-servers=https://192.168.51.2:2379"), but now
cty.StringVal("--etcd-servers=https://192.168.51.2:2379,https://192.168.51.3:2379").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for flexkube_controlplane.bootstrap to include new
values learned so far during apply, provider
"registry.terraform.io/-/flexkube" produced an invalid new value for
.state[0].config_files["/etc/kubernetes/kube-apiserver/pki/apiserver.crt"]:
was
cty.StringVal("384e9fb9988342d1f5e0b89d9acdb3baec454901ab06012136948915ddd7a84f"),
but now
cty.StringVal("2051c963372e209127d11ffb0d2fede01fe7d78fadc208cc804ea24e636ecc80").

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for flexkube_controlplane.bootstrap to include new
values learned so far during apply, provider
"registry.terraform.io/-/flexkube" produced an invalid new value for
.state_sensitive: inconsistent values for sensitive attribute.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for flexkube_controlplane.bootstrap to include new
values learned so far during apply, provider
"registry.terraform.io/-/flexkube" produced an invalid new value for
.state_yaml: inconsistent values for sensitive attribute.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.


Error: Provider produced inconsistent final plan

When expanding the plan for flexkube_controlplane.bootstrap to include new
values learned so far during apply, provider
"registry.terraform.io/-/flexkube" produced an invalid new value for
.config_yaml: inconsistent values for sensitive attribute.

This is a bug in the provider, which should be reported in the provider's own
issue tracker.

[invidian]

Except the error above, with upcoming fixes, adding new controller node works fine.

[invidian]

The same errors appears when removing new controller node. But running Terraform twice fixes it.