From 546d1d7835e42bb1fa1e5966c78137d7f559776b Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Tue, 16 Aug 2022 17:56:38 -0700 Subject: [PATCH 1/7] add blog post: immutability patterns using CEL --- .../2022-08-15-immutability-with-cel.md | 526 ++++++++++++++++++ 1 file changed, 526 insertions(+) create mode 100644 content/en/blog/_posts/2022-08-15-immutability-with-cel.md diff --git a/content/en/blog/_posts/2022-08-15-immutability-with-cel.md b/content/en/blog/_posts/2022-08-15-immutability-with-cel.md new file mode 100644 index 0000000000000..521c746c6a2fa --- /dev/null +++ b/content/en/blog/_posts/2022-08-15-immutability-with-cel.md @@ -0,0 +1,526 @@ +--- +layout: blog +title: "Enforce CRD Immutability with CEL Transition Rules" +date: 2022-08-15 +slug: enforce-immutability-using-cel +canonicalUrl: https://www.kubernetes.dev/blog/2022/08/15/enforce-immutability-using-cel +--- + +**Author:** [Alexander Zielenski (Google)](https://github.com/alexzielenski) + +--- + +# Introduction +Immutable fields can be found in a few places in the built-in Kubernetes types such as PodSpec, EphemeralContainers, Finalizers, and more. + +Until recently the best way to create immutable fields for Custom Resource Definitions has been to create an admission webhook: an unfortunate tradeoff for the common case of immutable fields. + +Beta since Kubernetes 1.25, CEL Validation Rules allow CRD authors to express validation constraints on their fields using a rich expression language, [CEL](https://github.com/google/cel-spec). This article explores how you can use validation rules to implement a few common immutability patterns. +# Validation Rules Basics +The new support for CEL validation rules in Kubernetes allows CRD authors to add complicated admission logic for their resources without writing any code! + +For example, A CEL rule to constrain a field `maximumSize` to be greater than a `minimumSize` for a CRD might look like the following: +```yaml + rule: self.maximumSize > self.minimumSize +message: “Maximum size must be greater than minimum size.” +``` +The rule field contains an expression written in CEL. `self` is a special keyword in CEL which refers to the object whose type contains the rule. + +The message field is an error message which will be sent to Kubernetes clients whenever this particular rule is not satisfied. +## More Information +For more details about the capabilities and limitations of Validation Rules using CEL, please refer to [the documentation](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules). The [cel-spec](https://github.com/google/cel-spec) is also a good reference for information specifically related to the language. + +# Immutability Patterns with CEL Validation Rules +This section implements several common use cases for immutability in Kubernetes Custom Resource Definitions using validation rules expressed as [kubebuilder marker comments](https://book.kubebuilder.io/reference/markers/crd.html). + +We will also show the resultant OpenAPI generated by the kubebuilder marker comments so that those not using kubebuilder can also follow along. + +## Project Setup + +To demonstrate how to use CEL rules with kubebuilder comments, we first need to set up a Golang project structure with the CRD defined in Go. Afterwards, we will be able to use this go project to demonstrate different design patterns. + +You may skip this step if you are only interested in the resultant OpenAPI extensions. + +We begin with a folder structure of a Go module set up like the following. If you have your own project already set up feel free to adapt this tutorial to your liking: +```console +cel-immutability-tutorial +├── generate.go +├── pkg +│ └── apis +│ └── stable.example.com +│ └── v1 +│ ├── doc.go +│ └── types.go +└── tools.go +``` + +This is the typical folder structure used by Kubernetes projects for defining new API resources. + +`doc.go` contains package-level metadata such as the group and the version: +```go +// +groupName=stable.example.com +// +versionName=v1 +package v1 +``` + +`types.go` contains all type definitions in stable.example.com/v1 + +```go +package v1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// An empty CRD as an example of defining a type using controller tools +// +kubebuilder:storageversion +// +kubebuilder:subresource:status +type TestCRD struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + Spec TestCRDSpec `json:"spec,omitempty"` + Status TestCRDStatus `json:"status,omitempty"` +} + +type TestCRDStatus struct {} +type TestCRDSpec struct { + // We will fill this in as we go along +} +``` + +`tools.go` contains a dependency on [controller-gen](https://book.kubebuilder.io/reference/generating-crd.html#generating-crds) which will be used to generate the CRD definition: + +```go +//go:build tools + +package celimmutabilitytutorial + +// Force direct dependency on code-generator so that we may use it with go run +import ( + _ "sigs.k8s.io/controller-tools/cmd/controller-gen" +) +``` + +Finally, `generate.go`contains a `go:generate` directive to make use of `controller-gen`. `controller-gen` parses our `types.go` and creates generates CRD yaml files into a `crd` folder: + +```go +package celimmutabilitytutorial + +//go:generate go run sigs.k8s.io/controller-tools/cmd/controller-gen crd paths=./pkg/apis/... output:dir=./crds +``` + + +You may now want to add dependencies for our definitions and test the code generation: + +```console +$ cd cel-immutability-tutorial +$ go mod init / +$ go mod tidy +$ go generate ./... +``` + +After running these commands we now have completed our basic project structure. The folder tree should look like this: + +``` +cel-immutability-tutorial +├── crds +│ └── stable.example.com_testcrds.yaml +├── generate.go +├── go.mod +├── go.sum +├── pkg +│ └── apis +│ └── stable.example.com +│ └── v1 +│ ├── doc.go +│ └── types.go +└── tools.go +``` + +Our test CRD yaml is now available in `crds/stable.example.com_testcrds.yaml` + +## Immutable After First Write +A common immutability design pattern is to make the field immutable once it has been first set. This example will throw a validation error if the field after changes after being first initialized. + +```go +// +kubebuilder:validation:XValidation:rule="!has(oldSelf.value) || has(self.value)", message="Value is required once set" +type ImmutableSinceFirstWrite struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +kubebuilder:validation:Optional + // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Value is immutable" + // +kubebuilder:validation:MaxLength=512 + Value string `json:"value"` +} +``` + +The `+kubebuilder` directives in the comments inform controller-gen how to annotate the generated OpenAPI. The `XValidation` rule causes the rule to appear among the `x-kubernetes-validations` OpenAPI extension. Kubernetes then respects the OpenAPI spec to enforce our constraints. + +To be immutable after first write, we require the following constraints: +1. Field must be allowed to be initially unset `+kubebuilder:validation:Optional` +2. Once set, field must not be allowed to be removed: `!has(oldSelf.value) | has(self.value)` (type-scoped rule) +3. Once set, field must not be allowed to change value `self == oldSelf` (field-scoped rule) + +Also note the additional directive `+kubebuilder:validation:MaxLength`. CEL requires that all strings have attached max length so that it may estimate the computation cost of the rule. Rules that are too expensive will be rejected. For more information on CEL cost budgeting, check out the other tutorial. + +### Example Usage + +```console +# Ensure the CRD yaml is generated by controller-gen +$ go generate ./... + +$ kubectl apply -f crds/stable.example.com_immutablesincefirstwrites.yaml +customresourcedefinition.apiextensions.k8s.io/immutablesincefirstwrites.stable.example.com created + +$ kubectl apply -f - <: Invalid value: "object": Value is required once set +``` + +### Generated openAPIV3Schema +Note that in the generated schema there are two separate rule locations. One is directly attached to the property `immutable_since_first_write`. The other rule is associated with the crd type itself. +```yaml +openAPIV3Schema: + properties: + value: + maxLength: 512 + type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf + type: object + x-kubernetes-validations: + - message: Value is required once set + rule: '!has(oldSelf.value) || has(self.value)' +``` + +## Immutable Upon Creation-Time + +A field which is immutable upon creation time is implemented similarly to the earlier example. The difference is that that +field is marked required, and the type-scoped rule is no longer necessary. + +```go +type ImmutableSinceCreation struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +kubebuilder:validation:Required + // +kubebuilder:validation:XValidation:rule="self == oldSelf",message="Value is immutable" + // +kubebuilder:validation:MaxLength=512 + Value string `json:"value"` +} +``` + +This field will be required when the object is created, and after that point will not be allowed to be modified. Our CEL Validation Rule `self == oldSelf` + +### Usage Example + +```console +# Ensure the CRD yaml is generated by controller-gen +$ go generate ./... + +$ kubectl apply -f crds/stable.example.com_immutablesincecreations.yaml +customresourcedefinition.apiextensions.k8s.io/immutablesincecreations.stable.example.com created + +$ kubectl apply -f - <: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation + +$ kubectl apply -f - <: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation +``` + +### Generated openAPIV3Schema +```yaml +openAPIV3Schema: + properties: + value: + maxLength: 512 + type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf + required: + - value + type: object +``` + +## Append-Only List of Containers +In the case of ephemeral containers on Pods, Kubernetes enforces that the elements in the list are immutable, and can’t be removed. The following shows how you could use CEL to achieve the same behavior. + +```go +// +kubebuilder:validation:XValidation:rule="!has(oldSelf.value) || has(self.value)", message="Value is required once set" +type AppendOnlyList struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +kubebuilder:validation:Optional + // +kubebuilder:validation:MaxItems=100 + // +kubebuilder:validation:XValidation:rule="oldSelf.all(x, x in self)",message="Values may only be added" + Values []v1.EphemeralContainer `json:"value"` +} +``` + +1. Once set, field must not be deleted: `!has(oldSelf.value) || has(self.value)` (type-scoped) +2. Once a value is added it is not removed: `oldSelf.all(x, x in self)` (field-scoped) +2. Value may be initially unset: `+kubebuilder:validation:Optional` + +Note that for cost-budgeting purposes, `MaxItems` is also required to be specified. + +### Example Usage + +```console +# Ensure the CRD yaml is generated by controller-gen +$ go generate ./... + +$ kubectl apply -f crds/stable.example.com_appendonlylists.yaml +customresourcedefinition.apiextensions.k8s.io/appendonlylists.stable.example.com created + +$ kubectl apply -f - <: Invalid value: "object": Value is required once set +``` + +### Generated openAPIV3Schema +```yaml +openAPIV3Schema: + properties: + value: + items: ... + maxItems: 100 + type: array + x-kubernetes-validations: + - message: Values may only be added + rule: oldSelf.all(x, x in self) + type: object + x-kubernetes-validations: + - message: Value is required once set + rule: '!has(oldSelf.value) || has(self.value)' +``` + +## Map with append-only keys, immutable values + +```go +// A map which does not allow keys to be removed or their values changed once set. New keys may be added, however. +// +kubebuilder:validation:XValidation:rule="!has(oldSelf.values) || has(self.values)", message="Value is required once set" +type MapAppendOnlyKeys struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // +kubebuilder:validation:Optional + // +kubebuilder:validation:MaxProperties=10 + // +kubebuilder:validation:XValidation:rule="oldSelf.all(key, key in self && self[key] == oldSelf[key])",message="Keys may not be removed and their values must stay the same" + Values map[string]string `json:"values,omitempty"` +} +``` + +1. Once set, field must not be deleted: `!has(oldSelf.values) || has(self.values)` (type-scoped) +2. Once a key is added it is not removed nor is its value modified: `oldSelf.all(key, key in self && self[key] == oldSelf[key])` (field-scoped) +3. Value may be initially unset: `+kubebuilder:validation:Optional` + +### Example Usage + +```console +# Ensure the CRD yaml is generated by controller-gen +$ go generate ./... + +$ kubectl apply -f crds/stable.example.com_mapappendonlykeys.yaml +customresourcedefinition.apiextensions.k8s.io/mapappendonlykeys.stable.example.com created + +$ kubectl apply -f - <: Invalid value: "object": Value is required once set +``` + +### Generated openAPIV3 Schema + +```yaml +openAPIV3Schema: + description: A map which does not allow keys to be removed or their values + changed once set. New keys may be added, however. + properties: + values: + additionalProperties: + type: string + maxProperties: 10 + type: object + x-kubernetes-validations: + - message: Keys may not be removed and their values must stay the same + rule: oldSelf.all(key, key in self && self[key] == oldSelf[key]) + type: object + x-kubernetes-validations: + - message: Value is required once set + rule: '!has(oldSelf.values) || has(self.values)' +``` + +# Going Further +The above examples showed how CEL rules can be added to kubebuilder types. The same rules can be added directly to OpenAPI if writing CRD yaml by hand. + +For native types, the same behavior can be achieved using kube-openapi’s marker [`+validations`](https://github.com/kubernetes/kube-openapi/blob/923526ac052c59656d41710b45bbcb03748aa9d6/pkg/generators/extension.go#L69). + +We have only scratched the surface of the expressiveness of validation rules using CEL. For more information please check out [the documentation](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules). From 358510fac1f102b82b3f3f526c02e0aff8f6427b Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Thu, 18 Aug 2022 11:52:40 -0700 Subject: [PATCH 2/7] account for tim's suggestions --- .../2022-08-15-immutability-with-cel.md | 348 ++++++++++++------ 1 file changed, 242 insertions(+), 106 deletions(-) diff --git a/content/en/blog/_posts/2022-08-15-immutability-with-cel.md b/content/en/blog/_posts/2022-08-15-immutability-with-cel.md index 521c746c6a2fa..5cceecaecd21b 100644 --- a/content/en/blog/_posts/2022-08-15-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-08-15-immutability-with-cel.md @@ -3,45 +3,75 @@ layout: blog title: "Enforce CRD Immutability with CEL Transition Rules" date: 2022-08-15 slug: enforce-immutability-using-cel -canonicalUrl: https://www.kubernetes.dev/blog/2022/08/15/enforce-immutability-using-cel --- -**Author:** [Alexander Zielenski (Google)](https://github.com/alexzielenski) +**Author:** [Alexander Zielenski](https://github.com/alexzielenski) (Google) --- +Immutable fields can be found in a few places in the built-in Kubernetes types. +For example, you can't change the `.metadata.name` of an object. Specific objects +have fields where changes to existing objects are constrained; for example, the +`.spec.selector` of a Deployment. -# Introduction -Immutable fields can be found in a few places in the built-in Kubernetes types such as PodSpec, EphemeralContainers, Finalizers, and more. +Aside from simple immutability, there are other common design patterns such as +lists which are append-only, or a map with mutable values and immutable keys. -Until recently the best way to create immutable fields for Custom Resource Definitions has been to create an admission webhook: an unfortunate tradeoff for the common case of immutable fields. +Until recently the best way to restrict field mutability for CustomResourceDefinitions +has been to create a validating +[admission webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-admission-webhooks): +this means a lot of complexity for the common case of making a field immutable. -Beta since Kubernetes 1.25, CEL Validation Rules allow CRD authors to express validation constraints on their fields using a rich expression language, [CEL](https://github.com/google/cel-spec). This article explores how you can use validation rules to implement a few common immutability patterns. -# Validation Rules Basics -The new support for CEL validation rules in Kubernetes allows CRD authors to add complicated admission logic for their resources without writing any code! +Beta since Kubernetes 1.25, CEL Validation Rules allow CRD authors to express +validation constraints on their fields using a rich expression language, +[CEL](https://github.com/google/cel-spec). This article explores how you can +use validation rules to implement a few common immutability patterns directly in +the manifest for a CRD. + +## Basics of validation rules + +The new support for CEL validation rules in Kubernetes allows CRD authors to add +complicated admission logic for their resources without writing any code! + +For example, A CEL rule to constrain a field `maximumSize` to be greater than a +`minimumSize` for a CRD might look like the following: -For example, A CEL rule to constrain a field `maximumSize` to be greater than a `minimumSize` for a CRD might look like the following: ```yaml - rule: self.maximumSize > self.minimumSize -message: “Maximum size must be greater than minimum size.” +rule: | + self.maximumSize > self.minimumSize +message: 'Maximum size must be greater than minimum size.' ``` -The rule field contains an expression written in CEL. `self` is a special keyword in CEL which refers to the object whose type contains the rule. -The message field is an error message which will be sent to Kubernetes clients whenever this particular rule is not satisfied. -## More Information -For more details about the capabilities and limitations of Validation Rules using CEL, please refer to [the documentation](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules). The [cel-spec](https://github.com/google/cel-spec) is also a good reference for information specifically related to the language. +The rule field contains an expression written in CEL. `self` is a special keyword +in CEL which refers to the object whose type contains the rule. -# Immutability Patterns with CEL Validation Rules -This section implements several common use cases for immutability in Kubernetes Custom Resource Definitions using validation rules expressed as [kubebuilder marker comments](https://book.kubebuilder.io/reference/markers/crd.html). +The message field is an error message which will be sent to Kubernetes clients +whenever this particular rule is not satisfied. -We will also show the resultant OpenAPI generated by the kubebuilder marker comments so that those not using kubebuilder can also follow along. +For more details about the capabilities and limitations of Validation Rules using +CEL, please refer to +[validation rules](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules). +The [CEL specification](https://github.com/google/cel-spec) is also a good +reference for information specifically related to the language. -## Project Setup +## Immutability patterns with CEL validation rules +This section implements several common use cases for immutability in Kubernetes +CustomResourceDefinitions, using validation rules expressed as +[kubebuilder marker comments](https://book.kubebuilder.io/reference/markers/crd.html). +Resultant OpenAPI generated by the kubebuilder marker comments will also be +included so that if you are writing your CRD manifests by hand you can still +follow along. -To demonstrate how to use CEL rules with kubebuilder comments, we first need to set up a Golang project structure with the CRD defined in Go. Afterwards, we will be able to use this go project to demonstrate different design patterns. +## Project setup -You may skip this step if you are only interested in the resultant OpenAPI extensions. +To use CEL rules with kubebuilder comments, you first need to set up a Golang +project structure with the CRD defined in Go. + +You may skip this step if you are not using kubebuilder or are only interested +in the resultant OpenAPI extensions. + +Begin with a folder structure of a Go module set up like the following. If +you have your own project already set up feel free to adapt this tutorial to your liking: -We begin with a folder structure of a Go module set up like the following. If you have your own project already set up feel free to adapt this tutorial to your liking: ```console cel-immutability-tutorial ├── generate.go @@ -85,7 +115,7 @@ type TestCRD struct { type TestCRDStatus struct {} type TestCRDSpec struct { - // We will fill this in as we go along + // You will fill this in as you go along } ``` @@ -96,13 +126,15 @@ type TestCRDSpec struct { package celimmutabilitytutorial -// Force direct dependency on code-generator so that we may use it with go run +// Force direct dependency on code-generator so that it may be executed with go run import ( _ "sigs.k8s.io/controller-tools/cmd/controller-gen" ) ``` -Finally, `generate.go`contains a `go:generate` directive to make use of `controller-gen`. `controller-gen` parses our `types.go` and creates generates CRD yaml files into a `crd` folder: +Finally, `generate.go`contains a `go:generate` directive to make use of +`controller-gen`. `controller-gen` parses our `types.go` and creates generates +CRD yaml files into a `crd` folder: ```go package celimmutabilitytutorial @@ -113,16 +145,17 @@ package celimmutabilitytutorial You may now want to add dependencies for our definitions and test the code generation: -```console -$ cd cel-immutability-tutorial -$ go mod init / -$ go mod tidy -$ go generate ./... +```shell +cd cel-immutability-tutorial +go mod init / +go mod tidy +go generate ./... ``` -After running these commands we now have completed our basic project structure. The folder tree should look like this: +After running these commands you now have completed the basic project structure. +Your folder tree should look like this: -``` +```console cel-immutability-tutorial ├── crds │ └── stable.example.com_testcrds.yaml @@ -138,10 +171,12 @@ cel-immutability-tutorial └── tools.go ``` -Our test CRD yaml is now available in `crds/stable.example.com_testcrds.yaml` +The manifest for the example CRD is now available in `crds/stable.example.com_testcrds.yaml`. -## Immutable After First Write -A common immutability design pattern is to make the field immutable once it has been first set. This example will throw a validation error if the field after changes after being first initialized. +## Immutablility after first modification +A common immutability design pattern is to make the field immutable once it has +been first set. This example will throw a validation error if the field after +changes after being first initialized. ```go // +kubebuilder:validation:XValidation:rule="!has(oldSelf.value) || has(self.value)", message="Value is required once set" @@ -156,35 +191,51 @@ type ImmutableSinceFirstWrite struct { } ``` -The `+kubebuilder` directives in the comments inform controller-gen how to annotate the generated OpenAPI. The `XValidation` rule causes the rule to appear among the `x-kubernetes-validations` OpenAPI extension. Kubernetes then respects the OpenAPI spec to enforce our constraints. +The `+kubebuilder` directives in the comments inform controller-gen how to +annotate the generated OpenAPI. The `XValidation` rule causes the rule to appear +among the `x-kubernetes-validations` OpenAPI extension. Kubernetes then +respects the OpenAPI spec to enforce our constraints. -To be immutable after first write, we require the following constraints: +To enforce a field's immutability after its first write, you need to apply the following constraints: 1. Field must be allowed to be initially unset `+kubebuilder:validation:Optional` 2. Once set, field must not be allowed to be removed: `!has(oldSelf.value) | has(self.value)` (type-scoped rule) 3. Once set, field must not be allowed to change value `self == oldSelf` (field-scoped rule) -Also note the additional directive `+kubebuilder:validation:MaxLength`. CEL requires that all strings have attached max length so that it may estimate the computation cost of the rule. Rules that are too expensive will be rejected. For more information on CEL cost budgeting, check out the other tutorial. +Also note the additional directive `+kubebuilder:validation:MaxLength`. CEL +requires that all strings have attached max length so that it may estimate the +computation cost of the rule. Rules that are too expensive will be rejected. +For more information on CEL cost budgeting, check out the other tutorial. -### Example Usage +### Example usage -```console +Generating and installing the CRD should succeed: +```shell # Ensure the CRD yaml is generated by controller-gen -$ go generate ./... - -$ kubectl apply -f crds/stable.example.com_immutablesincefirstwrites.yaml +go generate ./... +kubectl apply -f crds/stable.example.com_immutablesincefirstwrites.yaml +``` +```console customresourcedefinition.apiextensions.k8s.io/immutablesincefirstwrites.stable.example.com created +``` -$ kubectl apply -f - <: Invalid value: "object": Value is required once set ``` -### Generated openAPIV3Schema -Note that in the generated schema there are two separate rule locations. One is directly attached to the property `immutable_since_first_write`. The other rule is associated with the crd type itself. +### Generated schema +Note that in the generated schema there are two separate rule locations. +One is directly attached to the property `immutable_since_first_write`. +The other rule is associated with the crd type itself. + ```yaml openAPIV3Schema: properties: @@ -234,10 +299,11 @@ openAPIV3Schema: rule: '!has(oldSelf.value) || has(self.value)' ``` -## Immutable Upon Creation-Time +## Immutablility upon object creation -A field which is immutable upon creation time is implemented similarly to the earlier example. The difference is that that -field is marked required, and the type-scoped rule is no longer necessary. +A field which is immutable upon creation time is implemented similarly to the +earlier example. The difference is that that field is marked required, and the +type-scoped rule is no longer necessary. ```go type ImmutableSinceCreation struct { @@ -251,61 +317,84 @@ type ImmutableSinceCreation struct { } ``` -This field will be required when the object is created, and after that point will not be allowed to be modified. Our CEL Validation Rule `self == oldSelf` +This field will be required when the object is created, and after that point will +not be allowed to be modified. Our CEL Validation Rule `self == oldSelf` -### Usage Example +### Usage example -```console +Generating and installing the CRD should succeed: +```shell # Ensure the CRD yaml is generated by controller-gen -$ go generate ./... - -$ kubectl apply -f crds/stable.example.com_immutablesincecreations.yaml +go generate ./... +kubectl apply -f crds/stable.example.com_immutablesincecreations.yaml +``` +```console customresourcedefinition.apiextensions.k8s.io/immutablesincecreations.stable.example.com created +``` -$ kubectl apply -f - <: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation +``` -$ kubectl apply -f - <: Invalid value: "null": some validation rules were not checked because the object was invalid; correct the existing errors to complete validation ``` -### Generated openAPIV3Schema +### Generated schema ```yaml openAPIV3Schema: properties: @@ -320,8 +409,10 @@ openAPIV3Schema: type: object ``` -## Append-Only List of Containers -In the case of ephemeral containers on Pods, Kubernetes enforces that the elements in the list are immutable, and can’t be removed. The following shows how you could use CEL to achieve the same behavior. +## Append-only list of containers +In the case of ephemeral containers on Pods, Kubernetes enforces that the +elements in the list are immutable, and can’t be removed. The following example +shows how you could use CEL to achieve the same behavior. ```go // +kubebuilder:validation:XValidation:rule="!has(oldSelf.value) || has(self.value)", message="Value is required once set" @@ -342,16 +433,21 @@ type AppendOnlyList struct { Note that for cost-budgeting purposes, `MaxItems` is also required to be specified. -### Example Usage +### Example usage -```console +Generating and installing the CRD should succeed: +```shell # Ensure the CRD yaml is generated by controller-gen -$ go generate ./... - -$ kubectl apply -f crds/stable.example.com_appendonlylists.yaml +go generate ./... +kubectl apply -f crds/stable.example.com_appendonlylists.yaml +``` +```console customresourcedefinition.apiextensions.k8s.io/appendonlylists.stable.example.com created +``` -$ kubectl apply -f - <: Invalid value: "object": Value is required once set ``` -### Generated openAPIV3Schema +### Generated schema ```yaml openAPIV3Schema: properties: @@ -440,16 +551,20 @@ type MapAppendOnlyKeys struct { 2. Once a key is added it is not removed nor is its value modified: `oldSelf.all(key, key in self && self[key] == oldSelf[key])` (field-scoped) 3. Value may be initially unset: `+kubebuilder:validation:Optional` -### Example Usage - -```console +### Example usage +Generating and installing the CRD should succeed: +```shell # Ensure the CRD yaml is generated by controller-gen -$ go generate ./... - -$ kubectl apply -f crds/stable.example.com_mapappendonlykeys.yaml +go generate ./... +kubectl apply -f crds/stable.example.com_mapappendonlykeys.yaml +``` +```console customresourcedefinition.apiextensions.k8s.io/mapappendonlykeys.stable.example.com created +``` -$ kubectl apply -f - <: Invalid value: "object": Value is required once set ``` -### Generated openAPIV3 Schema +### Generated schema ```yaml openAPIV3Schema: @@ -518,9 +649,14 @@ openAPIV3Schema: rule: '!has(oldSelf.values) || has(self.values)' ``` -# Going Further -The above examples showed how CEL rules can be added to kubebuilder types. The same rules can be added directly to OpenAPI if writing CRD yaml by hand. +# Going further +The above examples showed how CEL rules can be added to kubebuilder types. +The same rules can be added directly to OpenAPI if writing a manifest for a CRD by hand. -For native types, the same behavior can be achieved using kube-openapi’s marker [`+validations`](https://github.com/kubernetes/kube-openapi/blob/923526ac052c59656d41710b45bbcb03748aa9d6/pkg/generators/extension.go#L69). +For native types, the same behavior can be achieved using kube-openapi’s marker +[`+validations`](https://github.com/kubernetes/kube-openapi/blob/923526ac052c59656d41710b45bbcb03748aa9d6/pkg/generators/extension.go#L69). -We have only scratched the surface of the expressiveness of validation rules using CEL. For more information please check out [the documentation](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules). +Usage of CEL within Kubernetes Validation Rules is so much more powerful than +what has been shown in this article. For more information please check out +[validation rules](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules) +in the Kubernetes documentation. From 2ea12c5a2b7833108b8aaf8682b98da6d647111a Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Mon, 22 Aug 2022 16:58:00 -0700 Subject: [PATCH 3/7] update publication date --- ...tability-with-cel.md => 2022-09-29-immutability-with-cel.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename content/en/blog/_posts/{2022-08-15-immutability-with-cel.md => 2022-09-29-immutability-with-cel.md} (99%) diff --git a/content/en/blog/_posts/2022-08-15-immutability-with-cel.md b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md similarity index 99% rename from content/en/blog/_posts/2022-08-15-immutability-with-cel.md rename to content/en/blog/_posts/2022-09-29-immutability-with-cel.md index 5cceecaecd21b..e80eee01a73b5 100644 --- a/content/en/blog/_posts/2022-08-15-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md @@ -1,7 +1,7 @@ --- layout: blog title: "Enforce CRD Immutability with CEL Transition Rules" -date: 2022-08-15 +date: 2022-09-29 slug: enforce-immutability-using-cel --- From 6c751ac3fb3e3457fa79cafd674af3f2386c6f30 Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Mon, 22 Aug 2022 17:19:48 -0700 Subject: [PATCH 4/7] replace folder diagrams with mermaid diagrams console output not portable --- .../2022-09-29-immutability-with-cel.md | 50 ++++++++----------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md index e80eee01a73b5..900fa6fe6c5b6 100644 --- a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md @@ -72,17 +72,14 @@ in the resultant OpenAPI extensions. Begin with a folder structure of a Go module set up like the following. If you have your own project already set up feel free to adapt this tutorial to your liking: -```console -cel-immutability-tutorial -├── generate.go -├── pkg -│ └── apis -│ └── stable.example.com -│ └── v1 -│ ├── doc.go -│ └── types.go -└── tools.go -``` +{{< mermaid >}} +graph LR + . --> generate.go + . --> pkg --> apis --> stable.example.com --> v1 + v1 --> doc.go + v1 --> types.go + . --> tools.go +{{}} This is the typical folder structure used by Kubernetes projects for defining new API resources. @@ -153,23 +150,20 @@ go generate ./... ``` After running these commands you now have completed the basic project structure. -Your folder tree should look like this: - -```console -cel-immutability-tutorial -├── crds -│ └── stable.example.com_testcrds.yaml -├── generate.go -├── go.mod -├── go.sum -├── pkg -│ └── apis -│ └── stable.example.com -│ └── v1 -│ ├── doc.go -│ └── types.go -└── tools.go -``` +Your folder tree should look like the following: + +{{< mermaid >}} +graph LR + . --> crds --> stable.example.com_testcrds.yaml + . --> generate.go + . --> go.mod + . --> go.sum + . --> pkg --> apis --> stable.example.com --> v1 + v1 --> doc.go + v1 --> types.go + . --> tools.go + +{{}} The manifest for the example CRD is now available in `crds/stable.example.com_testcrds.yaml`. From fccea057869d11da8727608752c2607d706d80b8 Mon Sep 17 00:00:00 2001 From: Alexander Zielenski <351783+alexzielenski@users.noreply.github.com> Date: Mon, 22 Aug 2022 17:22:53 -0700 Subject: [PATCH 5/7] add reference to other validation rules post --- content/en/blog/_posts/2022-09-29-immutability-with-cel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md index 900fa6fe6c5b6..1be807fd7444d 100644 --- a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md @@ -653,4 +653,4 @@ For native types, the same behavior can be achieved using kube-openapi’s marke Usage of CEL within Kubernetes Validation Rules is so much more powerful than what has been shown in this article. For more information please check out [validation rules](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules) -in the Kubernetes documentation. +in the Kubernetes documentation and [CRD Validation Rules Beta](https://kubernetes.io/blog/2022-09-23/crd-validation-rules-beta) blog post. From 26df0657956f410085d1765e2f1dd0e32f9870c1 Mon Sep 17 00:00:00 2001 From: Alex Zielenski Date: Mon, 26 Sep 2022 12:03:55 -0700 Subject: [PATCH 6/7] fix prior blog post url --- content/en/blog/_posts/2022-09-29-immutability-with-cel.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md index 1be807fd7444d..5308486fc8232 100644 --- a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md @@ -653,4 +653,4 @@ For native types, the same behavior can be achieved using kube-openapi’s marke Usage of CEL within Kubernetes Validation Rules is so much more powerful than what has been shown in this article. For more information please check out [validation rules](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules) -in the Kubernetes documentation and [CRD Validation Rules Beta](https://kubernetes.io/blog/2022-09-23/crd-validation-rules-beta) blog post. +in the Kubernetes documentation and [CRD Validation Rules Beta](https://kubernetes.io/blog/2022/09/23/crd-validation-rules-beta/) blog post. From 632156985a064dd55a9c7e2e09763914441fee4d Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Thu, 29 Sep 2022 15:00:01 +0100 Subject: [PATCH 7/7] Fix small snags --- content/en/blog/_posts/2022-09-29-immutability-with-cel.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md index 5308486fc8232..c857b06a1fbc3 100644 --- a/content/en/blog/_posts/2022-09-29-immutability-with-cel.md +++ b/content/en/blog/_posts/2022-09-29-immutability-with-cel.md @@ -7,7 +7,6 @@ slug: enforce-immutability-using-cel **Author:** [Alexander Zielenski](https://github.com/alexzielenski) (Google) ---- Immutable fields can be found in a few places in the built-in Kubernetes types. For example, you can't change the `.metadata.name` of an object. Specific objects have fields where changes to existing objects are constrained; for example, the @@ -293,7 +292,7 @@ openAPIV3Schema: rule: '!has(oldSelf.value) || has(self.value)' ``` -## Immutablility upon object creation +## Immutability upon object creation A field which is immutable upon creation time is implemented similarly to the earlier example. The difference is that that field is marked required, and the