From bb756bfca8cf8dd85f190abb3ded3310bc40308e Mon Sep 17 00:00:00 2001 From: Daniel Smith Date: Fri, 17 Nov 2017 16:05:32 -0800 Subject: [PATCH] Update admission control docs for webhooks --- docs/admin/admission-controllers.md | 221 ++++++++++++++++++---------- 1 file changed, 142 insertions(+), 79 deletions(-) diff --git a/docs/admin/admission-controllers.md b/docs/admin/admission-controllers.md index cdf99f30f95ef..26a5e2d4419a7 100644 --- a/docs/admin/admission-controllers.md +++ b/docs/admin/admission-controllers.md @@ -1,6 +1,6 @@ --- approvers: -- bprashanth +- lavalamp - davidopp - derekwaynecarr - erictune @@ -14,44 +14,62 @@ title: Using Admission Controllers ## What are they? -An admission control plug-in is a piece of code that intercepts requests to the Kubernetes -API server prior to persistence of the object, but after the request is authenticated -and authorized. The plug-in code is in the API server process -and must be compiled into the binary in order to be used at this time. - -Each admission control plug-in is run in sequence before a request is accepted into the cluster. If -any of the plug-ins in the sequence reject the request, the entire request is rejected immediately -and an error is returned to the end-user. - -Admission control plug-ins may mutate the incoming object in some cases to apply system configured -defaults. In addition, admission control plug-ins may mutate related resources as part of request -processing to do things like increment quota usage. +An admission controller is a piece of code that intercepts requests to the +Kubernetes API server prior to persistence of the object, but after the request +is authenticated and authorized. The controllers consist of the +[list](#what-does-each-admission-controller-do) below, are compiled into the +`kube-apiserver` binary, and may only be configured by the cluster +administrator. In that list, there are two special controllers: +MutatingAdmissionWebhook and ValidatingAdmissionWebhook. These execute the +mutating and validating (respectively) [admission control +webhooks](/docs/admin/extensible-admission-controllers.md#external-admission-webhooks) +which are configured in the API. + +Admission controllers may be "validating", "mutating", or both. Mutating +controllers may modify the objects they admit; validating controllers may not. + +The admission control process proceeds in two phases. In the first phase, +mutating admission controllers are run. In the second phase, validating +admission controllers are run. Note again that some of the controllers are +both. In both phases, the controllers are run in the order specified by the +`--admission-control` flag of `kube-apiserver`. + +If any of the controllers in either phase reject the request, the entire +request is rejected immediately and an error is returned to the end-user. + +Finally, in addition to sometimes mutating the object in question, admission +controllers may sometimes have side effects, that is, mutate related +resources as part of request processing. Incrementing quota usage is the +canonical example of why this is necessary. Any such side-effect needs a +corresponding reclamation or reconciliation process, as a given admission +controller does not know for sure that a given request will pass all of the +other admission controllers. ## Why do I need them? -Many advanced features in Kubernetes require an admission control plug-in to be enabled in order +Many advanced features in Kubernetes require an admission controller to be enabled in order to properly support the feature. As a result, a Kubernetes API server that is not properly -configured with the right set of admission control plug-ins is an incomplete server and will not +configured with the right set of admission controllers is an incomplete server and will not support all the features you expect. -## How do I turn on an admission control plug-in? +## How do I turn on an admission controller? The Kubernetes API server supports a flag, `admission-control` that takes a comma-delimited, ordered list of admission control choices to invoke prior to modifying objects in the cluster. -## What does each plug-in do? +## What does each admission controller do? ### AlwaysAdmit -Use this plugin by itself to pass-through all requests. +Use this admission controller by itself to pass-through all requests. ### AlwaysPullImages -This plug-in modifies every new Pod to force the image pull policy to Always. This is useful in a +This admission controller modifies every new Pod to force the image pull policy to Always. This is useful in a multitenant cluster so that users can be assured that their private images can only be used by those -who have the credentials to pull them. Without this plug-in, once an image has been pulled to a +who have the credentials to pull them. Without this admission controller, once an image has been pulled to a node, any pod from any user can use it simply by knowing the image's name (assuming the Pod is -scheduled onto the right node), without any authorization check against the image. When this plug-in +scheduled onto the right node), without any authorization check against the image. When this admission controller is enabled, images are always pulled prior to starting containers, which means valid credentials are required. @@ -61,22 +79,22 @@ Rejects all requests. Used for testing. ### DefaultStorageClass -This plug-in observes creation of `PersistentVolumeClaim` objects that do not request any specific storage class +This admission controller observes creation of `PersistentVolumeClaim` objects that do not request any specific storage class and automatically adds a default storage class to them. This way, users that do not request any special storage class do not need to care about them at all and they will get the default one. -This plug-in does not do anything when no default storage class is configured. When more than one storage +This admission controller does not do anything when no default storage class is configured. When more than one storage class is marked as default, it rejects any creation of `PersistentVolumeClaim` with an error and administrator must revisit `StorageClass` objects and mark only one as default. -This plugin ignores any `PersistentVolumeClaim` updates; it acts only on creation. +This admission controller ignores any `PersistentVolumeClaim` updates; it acts only on creation. See [persistent volume](/docs/concepts/storage/persistent-volumes/) documentation about persistent volume claims and storage classes and how to mark a storage class as default. ### DefaultTolerationSeconds -This plug-in sets the default forgiveness toleration for pods to tolerate +This admission controller sets the default forgiveness toleration for pods to tolerate the taints `notready:NoExecute` and `unreachable:NoExecute` for 5 minutes, if the pods don't already have toleration for taints `node.kubernetes.io/not-ready:NoExecute` or @@ -84,26 +102,26 @@ if the pods don't already have toleration for taints ### DenyExecOnPrivileged (deprecated) -This plug-in will intercept all requests to exec a command in a pod if that pod has a privileged container. +This admission controller will intercept all requests to exec a command in a pod if that pod has a privileged container. If your cluster supports privileged containers, and you want to restrict the ability of end-users to exec -commands in those containers, we strongly encourage enabling this plug-in. +commands in those containers, we strongly encourage enabling this admission controller. This functionality has been merged into [DenyEscalatingExec](#denyescalatingexec). ### DenyEscalatingExec -This plug-in will deny exec and attach commands to pods that run with escalated privileges that +This admission controller will deny exec and attach commands to pods that run with escalated privileges that allow host access. This includes pods that run as privileged, have access to the host IPC namespace, and have access to the host PID namespace. If your cluster supports containers that run with escalated privileges, and you want to restrict the ability of end-users to exec commands in those containers, we strongly encourage -enabling this plug-in. +enabling this admission controller. ### EventRateLimit (alpha) -This plug-in is introduced in v1.9 to mitigate the problem where the API server gets flooded by +This admission controller is introduced in v1.9 to mitigate the problem where the API server gets flooded by event requests. The cluster admin can specify event rate limits by: * Ensuring that `eventratelimit.admission.k8s.io/v1alpha1=true` is included in the @@ -137,18 +155,9 @@ EventRateLimit: See the [EventRateLimit proposal](https://git.k8s.io/community/contributors/design-proposals/api-machinery/admission_control_event_rate_limit.md) for more details. -### GenericAdmissionWebhook (alpha) - -This plug-in is related to the [Dynamic Admission Control](/docs/admin/extensible-admission-controllers) -introduced in v1.7. -The plug-in calls the webhooks configured via `ExternalAdmissionHookConfiguration`, -and only admits the operation if all the webhooks admit it. -Currently, the plug-in always fails open. -In other words, it ignores the failed calls to a webhook. - ### ImagePolicyWebhook -The ImagePolicyWebhook plug-in allows a backend webhook to make admission decisions. You enable this plug-in by setting the admission-control option as follows: +The ImagePolicyWebhook admission controller allows a backend webhook to make admission decisions. You enable this admission controller by setting the admission-control option as follows: ```shell --admission-control=ImagePolicyWebhook @@ -185,7 +194,7 @@ clusters: users: - name: name-of-api-server user: - client-certificate: /path/to/cert.pem # cert for the webhook plugin to use + client-certificate: /path/to/cert.pem # cert for the webhook admission controller to use client-key: /path/to/key.pem # key matching the cert ``` For additional HTTP configuration, refer to the [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) documentation. @@ -260,18 +269,18 @@ In any case, the annotations are provided by the user and are not validated by K ### Initializers (alpha) -This plug-in is introduced in v1.7. -The plug-in determines the initializers of a resource based on the existing +This admission controller is introduced in v1.7. +The admission controller determines the initializers of a resource based on the existing `InitializerConfiguration`s. It sets the pending initializers by modifying the metadata of the resource to be created. -For more information, please check [Dynamic Admission Control](/docs/admin/extensible-admission-controllers). +For more information, please check [Dynamic Admission Control](/docs/admin/extensible-admission-controllers.md). ### InitialResources (experimental) -This plug-in observes pod creation requests. If a container omits compute resource requests and limits, -then the plug-in auto-populates a compute resource request based on historical usage of containers running the same image. +This admission controller observes pod creation requests. If a container omits compute resource requests and limits, +then the admission controller auto-populates a compute resource request based on historical usage of containers running the same image. If there is not enough data to make a decision the Request is left unchanged. -When the plug-in sets a compute resource request, it does this by *annotating* the +When the admission controller sets a compute resource request, it does this by *annotating* the the pod spec rather than mutating the `container.resources` fields. The annotations added contain the information on what compute resources were auto-populated. @@ -279,69 +288,102 @@ See the [InitialResouces proposal](https://git.k8s.io/community/contributors/des ### LimitPodHardAntiAffinity -This plug-in denies any pod that defines `AntiAffinity` topology key other than +This admission controller denies any pod that defines `AntiAffinity` topology key other than `kubernetes.io/hostname` in `requiredDuringSchedulingRequiredDuringExecution`. ### LimitRanger -This plug-in will observe the incoming request and ensure that it does not violate any of the constraints +This admission controller will observe the incoming request and ensure that it does not violate any of the constraints enumerated in the `LimitRange` object in a `Namespace`. If you are using `LimitRange` objects in -your Kubernetes deployment, you MUST use this plug-in to enforce those constraints. LimitRanger can also +your Kubernetes deployment, you MUST use this admission controller to enforce those constraints. LimitRanger can also be used to apply default resource requests to Pods that don't specify any; currently, the default LimitRanger applies a 0.1 CPU requirement to all Pods in the `default` namespace. See the [limitRange design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_limit_range.md) and the [example of Limit Range](/docs/tasks/configure-pod-container/limit-range/) for more details. +### MutatingAdmissionWebhook (beta in 1.9) + +This admission controller calls any mutating webhooks which match the request. Matching +webhooks are called in serial; each one may modify the object if it desires. + +This admission controller (as implied by the name) only runs in the mutating phase. + +If a webhook called by this has side effects (for example, decrementing quota) it +*must* have a reconcilation system, as it is not guaranteed that subsequent +webhooks or validating admission controllers will permit the request to finish. + +If you disable the MutatingAdmissionWebhook, you must also disable the +`MutatingWebhookConfiguration` object in the `admissionregistration/v1beta1` +group/version via the `--runtime-config` flag (both are on by default in +versions >= 1.9). + +#### Use caution when authoring and installing mutating webhooks + + * Users may be confused when the objects they try to create are different from + what they get back. + * Setting originally unset fields is less confusing that overwriting fields set in + the request. Avoid doing the latter. + * Built in control loops may break when the objects they try to create are + different when read back. + * Setting originally unset fields is less likely to cause problems than + overwriting fields set in the original request. Avoid doing the latter. + * This is a beta feature. Future versions of Kubernetes may restrict the types of + mutations these webhooks can make. + * Future changes to control loops for built-in resources or third-party resources + may break webhooks that work well today. Even when the webhook installation API + is finalized, not all possible webhook behaviors will be guaranteed to be supported + indefinitely. + ### NamespaceAutoProvision -This plug-in examines all incoming requests on namespaced resources and checks +This admission controller examines all incoming requests on namespaced resources and checks if the referenced namespace does exist. It creates a namespace if it cannot be found. -This plug-in is useful in deployments that do not want to restrict creation of +This admission controller is useful in deployments that do not want to restrict creation of a namespace prior to its usage. ### NamespaceExists -This plug-in checks all requests on namespaced resources other than `Namespace` itself. +This admission controller checks all requests on namespaced resources other than `Namespace` itself. If the namespace referenced from a request doesn't exist, the request is rejected. ### NamespaceLifecycle -This plug-in enforces that a `Namespace` that is undergoing termination cannot have new objects created in it, -and ensures that requests in a non-existent `Namespace` are rejected. This plug-in also prevents deletion of +This admission controller enforces that a `Namespace` that is undergoing termination cannot have new objects created in it, +and ensures that requests in a non-existent `Namespace` are rejected. This admission controller also prevents deletion of three system reserved namespaces `default`, `kube-system`, `kube-public`. A `Namespace` deletion kicks off a sequence of operations that remove all objects (pods, services, etc.) in that -namespace. In order to enforce integrity of that process, we strongly recommend running this plug-in. +namespace. In order to enforce integrity of that process, we strongly recommend running this admission controller. ### NodeRestriction -This plug-in limits the `Node` and `Pod` objects a kubelet can modify. In order to be limited by this admission plugin, +This admission controller limits the `Node` and `Pod` objects a kubelet can modify. In order to be limited by this admission controller, kubelets must use credentials in the `system:nodes` group, with a username in the form `system:node:`. Such kubelets will only be allowed to modify their own `Node` API object, and only modify `Pod` API objects that are bound to their node. Future versions may add additional restrictions to ensure kubelets have the minimal set of permissions required to operate correctly. ### OwnerReferencesPermissionEnforcement -This plug-in protects the access to the `metadata.ownerReferences` of an object +This admission controller protects the access to the `metadata.ownerReferences` of an object so that only users with "delete" permission to the object can change it. -This plug-in also protects the access to `metadata.ownerReferences[x].blockOwnerDeletion` +This admission controller also protects the access to `metadata.ownerReferences[x].blockOwnerDeletion` of an object, so that only users with "update" permission to the `finalizers` subresource of the referenced *owner* can change it. ### PersistentVolumeLabel -This plug-in automatically attaches region or zone labels to PersistentVolumes -as defined by the cloud provider, e.g. GCE and AWS. +This admission controller automatically attaches region or zone labels to PersistentVolumes +as defined by the cloud provider (for example, GCE or AWS). It helps ensure the Pods and the PersistentVolumes mounted are in the same region and/or zone. -If the plug-in doesn't support automatic labelling your PersistentVolumes, you +If the admission controller doesn't support automatic labelling your PersistentVolumes, you may need to add the labels manually to prevent pods from mounting volumes from a different zone. ### PodNodeSelector -This plug-in defaults and limits what node selectors may be used within a namespace by reading a namespace annotation and a global configuration. +This admission controller defaults and limits what node selectors may be used within a namespace by reading a namespace annotation and a global configuration. #### Configuration File Format PodNodeSelector uses the admission config file `--admission-control-config-file` to set configuration options for the behavior of the backend. @@ -371,13 +413,13 @@ metadata: ### PersistentVolumeClaimResize -This plug-in implements additional validations for checking incoming `PersistentVolumeClaim` resize requests. +This admission controller implements additional validations for checking incoming `PersistentVolumeClaim` resize requests. **Note:** Support for volume resizing is available as an alpha feature. Admins must set the feature gate `ExpandPersistentVolumes` to `true` to enable resizing. {: .note} After enabling the `ExpandPersistentVolumes` feature gate, enabling the `PersistentVolumeClaimResize` admission -plug-in is recommended, too. This plug-in prevents resizing of all claims by default unless a claim's `StorageClass` +controller is recommended, too. This admission controller prevents resizing of all claims by default unless a claim's `StorageClass` explicitly enables resizing by setting `allowVolumeExpansion` to `true`. For example: all `PersistentVolumeClaim`s created from the following `StorageClass` support volume expansion: @@ -400,14 +442,14 @@ For more information about persistent volume claims, see ["PersistentVolumeClaim ### PodPreset -This plug-in injects a pod with the fields specified in a matching PodPreset. +This admission controller injects a pod with the fields specified in a matching PodPreset. See also [PodPreset concept](/docs/concepts/workloads/pods/podpreset/) and [Inject Information into Pods Using a PodPreset](/docs/tasks/inject-data-application/podpreset) for more information. ### PodSecurityPolicy -This plug-in acts on creation and modification of the pod and determines if it should be admitted +This admission controller acts on creation and modification of the pod and determines if it should be admitted based on the requested security context and the available Pod Security Policies. For Kubernetes < 1.6.0, the API Server must enable the extensions/v1beta1/podsecuritypolicy API @@ -418,7 +460,7 @@ for more information. ### PodTolerationRestriction -This plug-in first verifies any conflict between a pod's tolerations and its +This admission controller first verifies any conflict between a pod's tolerations and its namespace's tolerations, and rejects the pod request if there is a conflict. It then merges the namespace's tolerations into the pod's tolerations. The resulting tolerations are checked against the namespace's whitelist of @@ -440,47 +482,68 @@ The priority admission controller uses the `priorityClassName` field and populat ### ResourceQuota -This plug-in will observe the incoming request and ensure that it does not violate any of the constraints +This admission controller will observe the incoming request and ensure that it does not violate any of the constraints enumerated in the `ResourceQuota` object in a `Namespace`. If you are using `ResourceQuota` -objects in your Kubernetes deployment, you MUST use this plug-in to enforce quota constraints. +objects in your Kubernetes deployment, you MUST use this admission controller to enforce quota constraints. See the [resourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) and the [example of Resource Quota](/docs/concepts/policy/resource-quotas/) for more details. -It is strongly encouraged that this plug-in is configured last in the sequence of admission control plug-ins. This is +It is strongly encouraged that this admission controller is configured last in the sequence of admission controllers. This is so that quota is not prematurely incremented only for the request to be rejected later in admission control. ### SecurityContextDeny -This plug-in will deny any pod that attempts to set certain escalating [SecurityContext](/docs/user-guide/security-context) fields. This should be enabled if a cluster doesn't utilize [pod security policies](/docs/user-guide/pod-security-policy) to restrict the set of values a security context can take. +This admission controller will deny any pod that attempts to set certain escalating [SecurityContext](/docs/user-guide/security-context) fields. This should be enabled if a cluster doesn't utilize [pod security policies](/docs/user-guide/pod-security-policy) to restrict the set of values a security context can take. ### ServiceAccount -This plug-in implements automation for [serviceAccounts](/docs/user-guide/service-accounts). -We strongly recommend using this plug-in if you intend to make use of Kubernetes `ServiceAccount` objects. +This admission controller implements automation for [serviceAccounts](/docs/user-guide/service-accounts). +We strongly recommend using this admission controller if you intend to make use of Kubernetes `ServiceAccount` objects. + +### ValidatingAdmissionWebhook (alpha in 1.8; beta in 1.9) + +This admission controller calls any validating webhooks which match the request. Matching +webhooks are called in parallel; if any of them rejects the request, the request +fails. This admission controller only runs in the validation phase; the webhooks it calls may not +mutate the object, as opposed to the webhooks called by the `MutatingAdmissionWebhook` admission controller. + +If a webhook called by this has side effects (for example, decrementing quota) it +*must* have a reconcilation system, as it is not guaranteed that subsequent +webhooks or other validating admission controllers will permit the request to finish. +If you disable the ValidatingAdmissionWebhook, you must also disable the +`ValidatingWebhookConfiguration` object in the `admissionregistration/v1beta1` +group/version via the `--runtime-config` flag (both are on by default in +versions >= 1.9). -## Is there a recommended set of plug-ins to use? + +## Is there a recommended set of admission controllers to use? Yes. -For Kubernetes >= 1.6.0, we strongly recommend running the following set of admission control plug-ins (order matters): +For Kubernetes >= 1.9.0, we strongly recommend running the following set of admission controllers (order matters): + +```shell +--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ValidatingAdmissionWebhook,ResourceQuota,DefaultTolerationSeconds,MutatingAdmissionWebhook +``` +For Kubernetes >= 1.6.0, we strongly recommend running the following set of admission controllers (order matters): ```shell --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds ``` -For Kubernetes >= 1.4.0, we strongly recommend running the following set of admission control plug-ins (order matters): +For Kubernetes >= 1.4.0, we strongly recommend running the following set of admission controllers (order matters): ```shell --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota ``` -For Kubernetes >= 1.2.0, we strongly recommend running the following set of admission control plug-ins (order matters): +For Kubernetes >= 1.2.0, we strongly recommend running the following set of admission controllers (order matters): ```shell --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota ``` -For Kubernetes >= 1.0.0, we strongly recommend running the following set of admission control plug-ins (order matters): +For Kubernetes >= 1.0.0, we strongly recommend running the following set of admission controllers (order matters): ```shell --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,PersistentVolumeLabel,ResourceQuota