diff --git a/keps/sig-api-machinery/2887-openapi-enum-types/README.md b/keps/sig-api-machinery/2887-openapi-enum-types/README.md new file mode 100644 index 000000000000..f327de28141c --- /dev/null +++ b/keps/sig-api-machinery/2887-openapi-enum-types/README.md @@ -0,0 +1,830 @@ + +# KEP-NNNN: Your short, descriptive title + + + + + + +- [Release Signoff Checklist](#release-signoff-checklist) +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [Enum Pattern](#enum-pattern) + - [Enum Tag](#enum-tag) + - [Detect Possible Enum Types](#detect-possible-enum-types) + - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [OpenAPI v2 Breakage](#openapi-v2-breakage) + - [Test Plan](#test-plan) + - [Graduation Criteria](#graduation-criteria) + - [Alpha](#alpha) + - [Beta](#beta) + - [Stable](#stable) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) + - [Directly into OpenAPI v2](#directly-into-openapi-v2) + + +## Release Signoff Checklist + + + +Items marked with (R) are required *prior to targeting to a milestone / release*. + +- [X] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) +- [X] (R) KEP approvers have approved the KEP status as `implementable` +- [X] (R) Design details are appropriately documented +- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) + - [ ] e2e Tests for all Beta API Operations (endpoints) + - [ ] (R) Ensure GA e2e tests for meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [ ] (R) Graduation criteria is in place + - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) +- [ ] (R) Production readiness review completed +- [ ] (R) Production readiness review approved +- [ ] "Implementation History" section is up-to-date for milestone +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes + + + +[kubernetes.io]: https://kubernetes.io/ +[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes/kubernetes]: https://git.k8s.io/kubernetes +[kubernetes/website]: https://git.k8s.io/website + +## Summary + + + +As a sub-KEP of [KEP-2896 OpenAPI v3](https://github.com/kubernetes/enhancements/issues/2896), +this KEP defines Enum Types support for OpenAPI. Currently, types in our API have fields +that are actually enums but represented as plain string. This KEP proposes a `+enum` +annotation to type aliases that represent enums. The OpenAPI generator will have +the capability to recognize the annotation and auto-detect possible values for an enum. + +## Motivation + + + +Currently, types that are actually enums are represented as plain strings. For example, +here is the definition of `Protocol`, which must be one of `TCP`, `UDP`, and `SCTP`. +```go +package core +// ... + +// Protocol defines network protocols supported for things like container ports. +type Protocol string + +const ( + // ProtocolTCP is the TCP protocol. + ProtocolTCP Protocol = "TCP" + // ProtocolUDP is the UDP protocol. + ProtocolUDP Protocol = "UDP" + // ProtocolSCTP is the SCTP protocol. + ProtocolSCTP Protocol = "SCTP" +) + +// ContainerPort represents a network port in a single container +type ContainerPort struct { + // ... + + // Required: Supports "TCP", "UDP" and "SCTP" + // +optional + Protocol Protocol + + // ... +} +``` + +In order to convey that the `Protocol` type is an enum, every field of the type +have to comment its values like `ContainerPort` does. The same goes for validation - +protocol being one of "TCP", "UDP", and "SCTP" is duplicated among the code base. + +Tools and UIs can also benefit from the additional piece of information. For example, +Protocol can be shown as a dropdown of possible values instead of a textbox. +The user no longer needs to see an error of invalid value after submits the request. + +### Goals + + + +- Add an annotation that indicates a field as an enum. +- Make the Kubernetes OpenAPI generator recognize the annotation. +- Auto-detect possible values of an enum type. +- Detect and annotate enum types in all built-in types. + +### Non-Goals + + + +This KEP does not require removal of existing validation of enum types. While OpenAPI-based +of enum types can simplify this type of validation, retiring custom-made validations is out +of scope. + +This proposal does not intend to change the enum syntax for third-party tools like `kubebuilder`. +The implementation should only suggest changes to internal APIs. + +## Proposal + +As part of the OpenAPI v3 KEP, when generating OpenAPI schema, +the de facto enum types should be reflected as enums in the resulting schema. +We define enum types with following properties: +- All enum types are closed, i.e., the type includes a finite number of possible values. +- All enum types should be string. There can be no enum integers, floats, etc + +Additionally, This KEP makes the following assumptions on the implementation: +- All enum types have their own type alias. i.e. , no raw string enums. +- The canonical definition of a type is written in go and follows the standard + Kubernetes convention. + +### Enum Pattern + +The Enum Pattern of a type, appearing in a go source file, is defined as follows: +```go +package internal + +// FooType define sa type alias, underlying type must be string +type FooType string + +// in the same package, define constants with the alias +const ( + BarConst FooType = "Bar" + BazConst FooType = "Baz" +) + +// in one or more resources or nested objects, one of the field is of +// the defined alias type. + +// ExampleResource should refer FooType +type ExampleResource struct { + FooField FooType +} + +``` + +### Enum Tag +The Enum Tag is the annotation `// +enum` that must be in the closest comment blocks +of an enum type following Enum Pattern. + +```go +package internal + +// FooType define sa type alias, underlying type must be string +// +enum +type FooType string +``` + +An actual example looks like +```go +package core + +// Protocol defines network protocols supported for things like container ports. +// +enum +type Protocol string +``` + +### Detect Possible Enum Types + +A new API rule will be added to `kube-openapi` that warns about detected Enum Pattern +without the enum tag. If, in a rare situation, the detected type is not an enum, the user can +add an exception in `/api/api-rules/`. + + + +### Notes/Constraints/Caveats (Optional) + + + +Not all types following the Enum Pattern are actually enum types. Here is a counter-example. +```go +package core + +// StorageMedium defines ways that storage can be allocated to a volume. +type StorageMedium string + +// These are the valid value for StorageMedium +const ( + StorageMediumDefault StorageMedium = "" // use whatever the default is for the node + StorageMediumMemory StorageMedium = "Memory" // use memory (tmpfs) + StorageMediumHugePages StorageMedium = "HugePages" // use hugepages + StorageMediumHugePagesPrefix StorageMedium = "HugePages-" // prefix for full medium notation HugePages- +) +``` + +Here, `StorageMedium` can have infinite number of possible values, which disqualify it as an enum type. + +### Risks and Mitigations + + + +The syntax proposed in this KEP is different from existing implementation of kube-builder. +Namely, kube-builder requires possible values in the annotation itself, and is treated +as a validation. +```go +package example + +// Alias +// +kubebuilder:validation:Enum=Lion;Wolf;Dragon +type Alias string +``` +The two approaches are compatible, but eventually they will have to be unified. +However, compatibility with the ecosystem is out of scope of this proposal. + +## Design Details + + + +### OpenAPI v2 Breakage +This feature is not targeting OpenAPI v2 because it breaks existing code using +swagger to generate client or server stub from the spec. Take the following spec as +an example. + +```yaml +definitions: + Foo: + type: object + properties: + bar: + type: string + enum: + - A + - B + - C +``` + +The generated Java object looks like +```java +public class Foo { + + @JsonAdapter(BarEnum.Adapter.class) + public enum BarEnum { + A("A"), + B("B"), + C("C"); + // ... + } + private BarEnum bar = null; +} +``` +If the `enum` field was not in the definition, the `Bar` field would be a String. +This breaking change happens to all languages that have built-in enum support, +including Java, Kotlin, and TypeScript. + +### Test Plan + + + +Testing should focus on `kubernetes/kube-openapi` repository. There will be +integration tests against cases with valid and invalid definition of enum types. + +In the `kubernetes/kubernetes` repository, as part of OpenAPI v3 test suites, +there should be integration and e2e tests that validate present of enum fields. + +### Graduation Criteria + +#### Alpha + + - Feature implemented behind a feature gate, which shares with the main KEP + - Initial tests, both in k/k and k/kube-openapi, are completed and enabled + +#### Beta + + - All enums in Kubernetes built-in types have enum tags properly added. + +#### Stable + +This feature should graduate with OpenAPI v3 after beta. + +### Upgrade / Downgrade Strategy +Enable/disable the OpenAPI v3 feature gate. + +### Version Skew Strategy +An old version can still consume OpenAPI v2. For the conversion from v2 to v3 (see main KEP), +enum types will not present and thus should cause no issue. + +## Production Readiness Review Questionnaire + + + +### Feature Enablement and Rollback + + + +###### How can this feature be enabled / disabled in a live cluster? + + + +- [X] Feature gate (also fill in values in `kep.yaml`) + - Feature gate name: `OpenAPIv3Enabled` + - Components depending on the feature gate: `kube-apiserver` +- [ ] Other + - Describe the mechanism: + - Will enabling / disabling the feature require downtime of the control + plane? + - Will enabling / disabling the feature require downtime or reprovisioning + of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled). + +###### Does enabling the feature change any default behavior? + + + +No. This feature does not affect existing OpenAPI v2 generator. The tag in internal +APIs would be ignored by existing tooling. + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + + + +Yes. This feature can be turned off with OpenAPI v3. +###### What happens if we reenable the feature if it was previously rolled back? + +This feature needs access only to information immutable at run-time. There should +be no problem re-enabling it. +###### Are there any tests for feature enablement/disablement? + + +N/A +### Rollout, Upgrade and Rollback Planning + + + +###### How can a rollout or rollback fail? Can it impact already running workloads? + + + +###### What specific metrics should inform a rollback? + + + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + + + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + + + +### Monitoring Requirements + + + +###### How can an operator determine if the feature is in use by workloads? + + + +###### How can someone using this feature know that it is working for their instance? + + + +- [ ] Events + - Event Reason: +- [ ] API .status + - Condition name: + - Other field: +- [ ] Other (treat as last resort) + - Details: + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + + + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + + + +- [ ] Metrics + - Metric name: + - [Optional] Aggregation method: + - Components exposing the metric: +- [ ] Other (treat as last resort) + - Details: + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + + + +### Dependencies + + + +###### Does this feature depend on any specific services running in the cluster? + + + +### Scalability + + + +###### Will enabling / using this feature result in any new API calls? + + + +No. This feature only adds enum fields to enum types. +###### Will enabling / using this feature result in introducing new API types? + + + +No. +###### Will enabling / using this feature result in any new calls to the cloud provider? + + +No. +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + + +No. Enum information only appears in `/openapi/` endpoints but not standard APIs. +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + + +No. +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + + +No. The addition of enum fields take negligible amount of memory. +### Troubleshooting + + + +###### How does this feature react if the API server and/or etcd is unavailable? + +###### What are other known failure modes? + + + +###### What steps should be taken if SLOs are not being met to determine the problem? + +## Implementation History + + + +## Drawbacks + + +This KEP will break any users that expect enum types to be plain strings. +This is the main reason for not implementing this feature in OpenAPI v2 (see Alternative below). +## Alternatives + + + +### Directly into OpenAPI v2 + +The original design is to make the changes independent of OpenAPI v3. Doing this will +cause swagger-generated type to switch from plain string to a built-in enum, provided +the language supports enum type. This will break client code for many popular languages. +We can use a feature gate to give the maintainer some time to adapt the change, +but eventually the breakage will happen. diff --git a/keps/sig-api-machinery/2887-openapi-enum-types/kep.yaml b/keps/sig-api-machinery/2887-openapi-enum-types/kep.yaml new file mode 100644 index 000000000000..962e3ad9abd5 --- /dev/null +++ b/keps/sig-api-machinery/2887-openapi-enum-types/kep.yaml @@ -0,0 +1,40 @@ +title: KEP Template +kep-number: 2887 +authors: + - "@jiahuif" +owning-sig: sig-api-machinery +participating-sigs: [] +status: implementable +creation-date: 2021-08-20 +reviewers: + - '@jpbetz' + - '@apelisse' +approvers: + - '@jpbetz' + +see-also: + - https://github.com/kubernetes/enhancements/issues/2896 +replaces: [] + +# The target maturity stage in the current dev cycle for this KEP. +stage: alpha + +# The most recent milestone for which work toward delivery of this KEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v1.23" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + alpha: "v1.23" + +# The following PRR answers are required at alpha release +# List the feature gate name and the components for which it must be enabled +feature-gates: + - name: OpenAPIv3Enabled # inherited from the main KEP, #2896 + components: + - kube-apiserver +disable-supported: true + +# The following PRR answers are required at beta release +metrics: [] #TBD