Current consent of team Container

[itrich]

This discussion is intended to find a consent on the different topics we discussed during the last Team Container meetings.

[itrich]

1. Good implementation and/or defaults should avoid the need for custom annotations

[itrich]

@janiskemper: The high-level goal is to allow a generic deployement of applications to SCS-compliant clusters without the need to set custom annotations (for service objects) for each provider.

[itrich]

2. Where this is not possible, we should abstract differences away in the CCM; this differs from the current CCM approach somewhat

[janiskemper]

In the current CCM the load balancers are configured via annotations of the service objects, e.g. load-balancer.hetzner.cloud/id and loadbalancer.openstack.org/load-balancer-id. These annotations have a highly provider-specific naming. The goal is to standardize the naming for relevant inputs that are used by most providers. This can be implemented in different ways.

[itrich]

3. We only create mandatory opt-in features for things that are defined in k8s standards, such as ingress and metrics

[itrich]

As for now, we define this minimal set of mandatory opt-in features a provider needs to provide as:

• ingress
• metrics

[itrich]

The top priority should be to keep this list as minimal as possible!

[itrich]

4. Further (currently existing) opt-in features such as cert-manager, flux, harbor, ... are optional - avoid too high hurdles

[itrich]

We should not provide a list of optional opt-in features!

[itrich]

Optional opt-in features can – of course – be implemented in the reference implementation (and thus making it easier for providers to provide usefull features)

[itrich]

5. We need to continue with defining a set of standard settings (clusterctl.yaml / values.yaml) that define the cluster with its standard features. This is the set of cluster-API parameters, generalized a bit where needed to allow for non-CAPI implementations

[itrich]

Idea: Schedule a workshop (2-3h) to define this list of standard settings. This should be the last step in defining the SCS Container standard.

[garloff]

Workshop next Mon 15 -- 17:xx

[itrich]

6. We had consensus that network policies need to be supported; whether or not we need to specify the CNI implementation (with a preference for cilium) is open

[itrich]

Let us discuss next week what it means to support network policies.

[ajfriesen]

We should not specify a specific CNI provider.
Instead, we should just specify features which the CNI should implement, like the network policies

[garloff]

k8s network policies are a defined thing.
There is also the possibility to use cilium specific network policies by changing the CRDs.
How to deal with it in the ref impl:

1. Disallow by default, allow opt-in
2. Allow, but warn
3. Provide external suite to ensure that the non-portability can be detected

[garloff]

Best approach seems to be no (2) => create validating webhook (API group: admission registration).
https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/

Generic approach: Warnings for CRDs that come from the provider but that are not part of the standard.

[garloff]

--dry-run=server would call the API server and (hopefully) trigger the validating webhooks, so the tool to test portability would not need separate checks but just use the webhooks.

[garloff]

"SCS - compatible helm charts":
Idea: Implement compatibility check

• We would check that no non-standard features are being used there and fail if they are.

[garloff]

Implementation: helm dry-run => look at deployed resources and analyze them.
Challenge: This validates it for one set of parameters chosen.
(Could tests with a set of values.yaml files to cover more -- complete coverage will be still tough in real-world scenarios.)

[garloff]

Other option: Look at https://github.com/helm/chart-testing

[garloff]

How to deal with v1beta1 things?

• We can not guarantee this in the future (as CNCF does not guarantee this)
• On the other hand, people do use v1beta1 things -> not our job to reject this
• If we create a checker tool, could allow a switch to show warnings for these (opt-in!)

[garloff]

This is about the future-proofness (and not about the compatibility b/w different SCS implementations) and thus not our main focus (as it's a generic CNCF thing).