diff --git a/docs/tasks/administer-cluster/kubelet-config-file.md b/docs/tasks/administer-cluster/kubelet-config-file.md index eea9ee9464489..1cb5c27757a67 100644 --- a/docs/tasks/administer-cluster/kubelet-config-file.md +++ b/docs/tasks/administer-cluster/kubelet-config-file.md @@ -6,18 +6,20 @@ title: Set Kubelet parameters via a config file --- {% capture overview %} -{% include feature-state-alpha.md %} +{% include feature-state-beta.md %} -As of Kubernetes 1.8, a subset of the Kubelet's configuration parameters may be -set via an on-disk config file, as a substitute for command-line flags. In the -future, most of the existing command-line flags will be deprecated in favor of -providing parameters via a config file, which simplifies node deployment. +A subset of the Kubelet's configuration parameters may be +set via an on-disk config file, as a substitute for command-line flags. +This functionality is considered beta in v1.10. + +Providing parameters via a config file is the recommended approach because +it simplifies node deployment and configuration management. {% endcapture %} {% capture prerequisites %} -- A v1.8 or higher Kubelet binary must be installed. +- A v1.10 or higher Kubelet binary must be installed for beta functionality. {% endcapture %} @@ -27,24 +29,42 @@ providing parameters via a config file, which simplifies node deployment. The subset of the Kubelet's configuration that can be configured via a file is defined by the `KubeletConfiguration` struct -[here (v1alpha1)](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/apis/kubeletconfig/v1alpha1/types.go). +[here (v1beta1)](https://github.com/kubernetes/kubernetes/blob/release-1.10/pkg/kubelet/apis/kubeletconfig/v1beta1/types.go). + The configuration file must be a JSON or YAML representation of the parameters -in this struct. Note that this structure, and thus the config file API, -is still considered alpha and is not subject to stability guarantees. +in this struct. Make sure the Kubelet has read permissions on the file. + +Here is an example of what this file might look like: +``` +kind: KubeletConfiguration +apiVersion: kubeletconfig/v1beta1 +evictionHard: + memory.available: "200Mi" +``` -Create a file named `kubelet` in its own directory and make sure the directory -and file are both readable by the Kubelet. You should write your intended -Kubelet configuration in this `kubelet` file. +In the example, the Kubelet is configured to evict Pods when available memory drops below 200Mi. +All other Kubelet configuration values are left at their built-in defaults, unless overridden +by flags. Command line flags which target the same value as a config file will override that value. For a trick to generate a configuration file from a live node, see [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet). ## Start a Kubelet process configured via the config file -Start the Kubelet with the `--init-config-dir` flag set to the location of the directory -containing the `kubelet` file. The Kubelet will then load the parameters defined -by `KubeletConfiguration` from the `kubelet` file, rather than from their -associated command-line flags. +Start the Kubelet with the `--config` flag set to the path of the Kubelet's config file. +The Kubelet will then load its config from this file. + +Note that command line flags which target the same value as a config file will override that value. +This helps ensure backwards compatibility with the command-line API. + +Note that relative file paths in the Kubelet config file are resolved relative to the +location of the Kubelet config file, whereas relative paths in command line flags are resolved +relative to the Kubelet's current working directory. + +Note that some default values differ between command-line flags and the Kubelet config file. +If `--config` is provided and the values are not specified via the command line, the +defaults for the `KubeletConfiguration` version apply. +In the above example, this version is `kubeletconfig/v1beta1`. {% endcapture %} @@ -53,16 +73,8 @@ associated command-line flags. ## Relationship to Dynamic Kubelet Config If you are using the [Dynamic Kubelet Configuration](/docs/tasks/administer-cluster/reconfigure-kubelet) -feature, the configuration provided via `--init-config-dir` will be considered -the "last known good" configuration by the automatic rollback mechanism. - -Note that the layout of the files in the `--init-config-dir` mirrors the layout -of data in the ConfigMaps used for Dynamic Kubelet Config; the file names are -the same as the keys of the ConfigMap, and the file contents are JSON or YAML -representations of the same structures. Today, the only pair is -`kubelet:KubeletConfiguration`, though more may emerge in the future. -See [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet) -for more information. +feature, the combination of configuration provided via `--config` and any flags which override these values +is considered the default "last known good" configuration by the automatic rollback mechanism. {% endcapture %} diff --git a/docs/tasks/administer-cluster/reconfigure-kubelet.md b/docs/tasks/administer-cluster/reconfigure-kubelet.md index 90307c46a7b4f..83e55255b3b26 100644 --- a/docs/tasks/administer-cluster/reconfigure-kubelet.md +++ b/docs/tasks/administer-cluster/reconfigure-kubelet.md @@ -45,18 +45,18 @@ Kubelet's configuration. 3. Update the Kubelet's correspoinding Node object to use this ConfigMap. Each Kubelet watches a configuration reference on its respective Node object. -When this reference changes, the Kubelet downloads the new configuration and -exits. For the feature to work correctly, you must be running a process manager +When this reference changes, the Kubelet downloads the new configuration, +updates a local reference to refer to the file, and exits. +For the feature to work correctly, you must be running a process manager (like systemd) which will restart the Kubelet when it exits. When the Kubelet is restarted, it will begin using the new configuration. -The new configuration completely overrides the old configuration; unspecified -fields in the new configuration will receive their canonical default values. -Some CLI flags do not have an associated configuration field, and will not be -affected by the new configuration. These fields are defined by the KubeletFlags -structure, [here](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go). +The new configuration completely overrides configuration provided by `--config`, +and is overridden by command-line flags. Unspecified values in the new configuration +will receive default values appropriate to the configuration version +(e.g. `kubeletconfig/v1beta1`), unless overridden by flags. -The status of the Node's Kubelet configuration is reported via the `ConfigOK` +The status of the Node's Kubelet configuration is reported via the `KubeletConfigOK` condition in the Node status. Once you have updated a Node to use the new ConfigMap, you can observe this condition to confirm that the Node is using the intended configuration. A table describing the possible conditions can be found @@ -95,13 +95,13 @@ and you will simply edit a copy of this file (which, as a best practice, should live in version control) while creating the first Kubelet ConfigMap. Today, however, the Kubelet is still bootstrapped with command-line flags. Fortunately, there is a dirty trick you can use to generate a config file containing a Node's -current configuration. The trick involves hitting the Kubelet server's `configz` +current configuration. The trick involves accessing the Kubelet server's `configz` endpoint via the kubectl proxy. This endpoint, in its current implementation, is intended to be used only as a debugging aid, which is part of why this is a -dirty trick. There is ongoing work to improve the endpoint, and in the future -this will be a less "dirty" operation. This trick also requires the `jq` command -to be installed on your machine, for unpacking and editing the JSON response -from the endpoint. +dirty trick. The endpoint may be improved in the future, but until then +it should not be relied on for production scenarios. +This trick also requires the `jq` command to be installed on your machine, +for unpacking and editing the JSON response from the endpoint. Do the following to generate the file: @@ -112,12 +112,12 @@ configz endpoint: ``` $ export NODE_NAME=the-name-of-the-node-you-are-reconfiguring -$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1alpha1"' > kubelet_configz_${NODE_NAME} +$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1beta1"' > kubelet_configz_${NODE_NAME} ``` Note that we have to manually add the `kind` and `apiVersion` to the downloaded object, as these are not reported by the configz endpoint. This is one of the -limitations of the endpoint that is planned to be fixed in the future. +limitations of the endpoint. ### Edit the configuration file @@ -209,29 +209,29 @@ Be sure to specify all three of `name`, `namespace`, and `uid`. ### Observe that the Node begins using the new configuration Retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and look for the -`ConfigOK` condition in `status.conditions`. You should see the message +`KubeletConfigOK` condition in `status.conditions`. You should see the message `Using current (UID: CONFIG_MAP_UID)` when the Kubelet starts using the new configuration. For convenience, you can use the following command (using `jq`) to filter down -to the `ConfigOK` condition: +to the `KubeletConfigOK` condition: ``` -$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="ConfigOK"))' +$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="KubeletConfigOK"))' [ { "lastHeartbeatTime": "2017-09-20T18:08:29Z", "lastTransitionTime": "2017-09-20T18:08:17Z", - "message": "using current (UID: \"2ebc8d1a-9e2a-11e7-a8dd-42010a800006\")", + "message": "using current: /api/v1/namespaces/kube-system/configmaps/my-node-config-gkt4c2m4b2", "reason": "passing all checks", "status": "True", - "type": "ConfigOK" + "type": "KubeletConfigOK" } ] ``` If something goes wrong, you may see one of several different error conditions, -detailed in the Table of ConfigOK Conditions, below. When this happens, you +detailed in the table of KubeletConfigOK conditions, below. When this happens, you should check the Kubelet's log for more details. ### Edit the configuration file again @@ -282,16 +282,16 @@ the following, with `name` and `uid` substituted as necessary: ``` configSource: configMapRef: - name: NEW_CONFIG_MAP_NAME + name: ${NEW_CONFIG_MAP_NAME} namespace: kube-system - uid: NEW_CONFIG_MAP_UID + uid: ${NEW_CONFIG_MAP_UID} ``` ### Observe that the Kubelet is using the new configuration Once more, retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and -look for the `ConfigOK` condition in `status.conditions`. You should the message -`Using current (UID: NEW_CONFIG_MAP_UID)` when the Kubelet starts using the +look for the `KubeletConfigOK` condition in `status.conditions`. You should see the message +`using current: /api/v1/namespaces/kube-system/configmaps/${NEW_CONFIG_MAP_NAME}` when the Kubelet starts using the new configuration. ### Deauthorize your Node fom reading the old ConfigMap @@ -327,9 +327,8 @@ remove the `spec.configSource` subfield. ### Observe that the Node is using its local default configuration -After removing this subfield, you should eventually observe that the ConfigOK -condition's message reverts to either `using current (default)` or -`using current (init)`, depending on how the Node was provisioned. +After removing this subfield, you should eventually observe that the KubeletConfigOK +condition's message reverts to `using current: local`. ### Deauthorize your Node fom reading the old ConfigMap @@ -366,9 +365,9 @@ Here is an example command that uses `kubectl patch`: kubectl patch node ${NODE_NAME} -p "{\"spec\":{\"configSource\":{\"configMapRef\":{\"name\":\"${CONFIG_MAP_NAME}\",\"namespace\":\"kube-system\",\"uid\":\"${CONFIG_MAP_UID}\"}}}}" ``` -## Understanding ConfigOK Conditions +## Understanding KubeletConfigOK Conditions -The following table describes several of the `ConfigOK` Node conditions you +The following table describes several of the `KubeletConfigOK` Node conditions you might encounter in a cluster that has Dynamic Kubelet Config enabled. If you observe a condition with `status=False`, you should check the Kubelet log for more error details by searching for the message or reason text. @@ -383,49 +382,33 @@ more error details by searching for the message or reason text. Status -

using current (default)

-

current is set to the local default, and no init config was provided

+

using current: local

+

when the config source is nil, the Kubelet uses its local config

True

-

using current (init)

-

current is set to the local default, and an init config was provided

-

True

- - -

using current (UID: CURRENT_CONFIG_MAP_UID)

+

using current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}

passing all checks

True

-

using last-known-good (default)

- - - -

False

- - -

using last-known-good (init)

+

using last-known-good: local

False

-

using last-known-good (UID: LAST_KNOWN_GOOD_CONFIG_MAP_UID)

+

using last-known-good: /api/v1/namespaces/${LAST_KNOWN_GOOD_CONFIG_MAP_NAMESPACE}/configmaps/${LAST_KNOWN_GOOD_CONFIG_MAP_NAME}

False

@@ -451,15 +434,15 @@ more error details by searching for the message or reason text.

failed to sync, reason:

False