Skip to content

Commit

Permalink
update kubeletconfig docs for v1.10, beta
Browse files Browse the repository at this point in the history
  • Loading branch information
@mtaufen
mtaufen committed Mar 1, 2018
1 parent 857fee8 commit f5ae314
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 90 deletions.
64 changes: 38 additions & 26 deletions docs/tasks/administer-cluster/kubelet-config-file.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,20 @@ title: Set Kubelet parameters via a config file
---

{% capture overview %}
{% include feature-state-alpha.md %}
{% include feature-state-beta.md %}

As of Kubernetes 1.8, a subset of the Kubelet's configuration parameters may be
set via an on-disk config file, as a substitute for command-line flags. In the
future, most of the existing command-line flags will be deprecated in favor of
providing parameters via a config file, which simplifies node deployment.
A subset of the Kubelet's configuration parameters may be
set via an on-disk config file, as a substitute for command-line flags.
This functionality is considered beta in v1.10.

Providing parameters via a config file is the recommended approach because
it simplifies node deployment and configuration management.

{% endcapture %}

{% capture prerequisites %}

- A v1.8 or higher Kubelet binary must be installed.
- A v1.10 or higher Kubelet binary must be installed for beta functionality.

{% endcapture %}

Expand All @@ -27,24 +29,42 @@ providing parameters via a config file, which simplifies node deployment.

The subset of the Kubelet's configuration that can be configured via a file
is defined by the `KubeletConfiguration` struct
[here (v1alpha1)](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/apis/kubeletconfig/v1alpha1/types.go).
[here (v1beta1)](https://github.com/kubernetes/kubernetes/blob/release-1.10/pkg/kubelet/apis/kubeletconfig/v1beta1/types.go).

The configuration file must be a JSON or YAML representation of the parameters
in this struct. Note that this structure, and thus the config file API,
is still considered alpha and is not subject to stability guarantees.
in this struct. Make sure the Kubelet has read permissions on the file.

Here is an example of what this file might look like:
```
kind: KubeletConfiguration
apiVersion: kubeletconfig/v1beta1
evictionHard:
memory.available: "200Mi"
```

Create a file named `kubelet` in its own directory and make sure the directory
and file are both readable by the Kubelet. You should write your intended
Kubelet configuration in this `kubelet` file.
In the example, the Kubelet is configured to evict Pods when available memory drops below 200Mi.
All other Kubelet configuration values are left at their built-in defaults, unless overridden
by flags. Command line flags which target the same value as a config file will override that value.

For a trick to generate a configuration file from a live node, see
[Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet).

## Start a Kubelet process configured via the config file

Start the Kubelet with the `--init-config-dir` flag set to the location of the directory
containing the `kubelet` file. The Kubelet will then load the parameters defined
by `KubeletConfiguration` from the `kubelet` file, rather than from their
associated command-line flags.
Start the Kubelet with the `--config` flag set to the path of the Kubelet's config file.
The Kubelet will then load its config from this file.

Note that command line flags which target the same value as a config file will override that value.
This helps ensure backwards compatibility with the command-line API.

Note that relative file paths in the Kubelet config file are resolved relative to the
location of the Kubelet config file, whereas relative paths in command line flags are resolved
relative to the Kubelet's current working directory.

Note that some default values differ between command-line flags and the Kubelet config file.
If `--config` is provided and the values are not specified via the command line, the
defaults for the `KubeletConfiguration` version apply.
In the above example, this version is `kubeletconfig/v1beta1`.

{% endcapture %}

Expand All @@ -53,16 +73,8 @@ associated command-line flags.
## Relationship to Dynamic Kubelet Config

If you are using the [Dynamic Kubelet Configuration](/docs/tasks/administer-cluster/reconfigure-kubelet)
feature, the configuration provided via `--init-config-dir` will be considered
the "last known good" configuration by the automatic rollback mechanism.

Note that the layout of the files in the `--init-config-dir` mirrors the layout
of data in the ConfigMaps used for Dynamic Kubelet Config; the file names are
the same as the keys of the ConfigMap, and the file contents are JSON or YAML
representations of the same structures. Today, the only pair is
`kubelet:KubeletConfiguration`, though more may emerge in the future.
See [Reconfigure a Node's Kubelet in a Live Cluster](/docs/tasks/administer-cluster/reconfigure-kubelet)
for more information.
feature, the combination of configuration provided via `--config` and any flags which override these values
is considered the default "last known good" configuration by the automatic rollback mechanism.

{% endcapture %}

Expand Down
111 changes: 47 additions & 64 deletions docs/tasks/administer-cluster/reconfigure-kubelet.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,18 +45,18 @@ Kubelet's configuration.
3. Update the Kubelet's correspoinding Node object to use this ConfigMap.

Each Kubelet watches a configuration reference on its respective Node object.
When this reference changes, the Kubelet downloads the new configuration and
exits. For the feature to work correctly, you must be running a process manager
When this reference changes, the Kubelet downloads the new configuration,
updates a local reference to refer to the file, and exits.
For the feature to work correctly, you must be running a process manager
(like systemd) which will restart the Kubelet when it exits. When the Kubelet is
restarted, it will begin using the new configuration.

The new configuration completely overrides the old configuration; unspecified
fields in the new configuration will receive their canonical default values.
Some CLI flags do not have an associated configuration field, and will not be
affected by the new configuration. These fields are defined by the KubeletFlags
structure, [here](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go).
The new configuration completely overrides configuration provided by `--config`,
and is overridden by command-line flags. Unspecified values in the new configuration
will receive default values appropriate to the configuration version
(e.g. `kubeletconfig/v1beta1`), unless overridden by flags.

The status of the Node's Kubelet configuration is reported via the `ConfigOK`
The status of the Node's Kubelet configuration is reported via the `KubeletConfigOK`
condition in the Node status. Once you have updated a Node to use the new
ConfigMap, you can observe this condition to confirm that the Node is using the
intended configuration. A table describing the possible conditions can be found
Expand Down Expand Up @@ -95,13 +95,13 @@ and you will simply edit a copy of this file (which, as a best practice, should
live in version control) while creating the first Kubelet ConfigMap. Today,
however, the Kubelet is still bootstrapped with command-line flags. Fortunately,
there is a dirty trick you can use to generate a config file containing a Node's
current configuration. The trick involves hitting the Kubelet server's `configz`
current configuration. The trick involves accessing the Kubelet server's `configz`
endpoint via the kubectl proxy. This endpoint, in its current implementation, is
intended to be used only as a debugging aid, which is part of why this is a
dirty trick. There is ongoing work to improve the endpoint, and in the future
this will be a less "dirty" operation. This trick also requires the `jq` command
to be installed on your machine, for unpacking and editing the JSON response
from the endpoint.
dirty trick. The endpoint may be improved in the future, but until then
it should not be relied on for production scenarios.
This trick also requires the `jq` command to be installed on your machine,
for unpacking and editing the JSON response from the endpoint.

Do the following to generate the file:

Expand All @@ -112,12 +112,12 @@ configz endpoint:

```
$ export NODE_NAME=the-name-of-the-node-you-are-reconfiguring
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1alpha1"' > kubelet_configz_${NODE_NAME}
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1beta1"' > kubelet_configz_${NODE_NAME}
```

Note that we have to manually add the `kind` and `apiVersion` to the downloaded
object, as these are not reported by the configz endpoint. This is one of the
limitations of the endpoint that is planned to be fixed in the future.
limitations of the endpoint.

### Edit the configuration file

Expand Down Expand Up @@ -209,29 +209,29 @@ Be sure to specify all three of `name`, `namespace`, and `uid`.
### Observe that the Node begins using the new configuration

Retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and look for the
`ConfigOK` condition in `status.conditions`. You should see the message
`KubeletConfigOK` condition in `status.conditions`. You should see the message
`Using current (UID: CONFIG_MAP_UID)` when the Kubelet starts using the new
configuration.

For convenience, you can use the following command (using `jq`) to filter down
to the `ConfigOK` condition:
to the `KubeletConfigOK` condition:

```
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="ConfigOK"))'
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="KubeletConfigOK"))'
[
{
"lastHeartbeatTime": "2017-09-20T18:08:29Z",
"lastTransitionTime": "2017-09-20T18:08:17Z",
"message": "using current (UID: \"2ebc8d1a-9e2a-11e7-a8dd-42010a800006\")",
"message": "using current: /api/v1/namespaces/kube-system/configmaps/my-node-config-gkt4c2m4b2",
"reason": "passing all checks",
"status": "True",
"type": "ConfigOK"
"type": "KubeletConfigOK"
}
]
```

If something goes wrong, you may see one of several different error conditions,
detailed in the Table of ConfigOK Conditions, below. When this happens, you
detailed in the table of KubeletConfigOK conditions, below. When this happens, you
should check the Kubelet's log for more details.

### Edit the configuration file again
Expand Down Expand Up @@ -282,16 +282,16 @@ the following, with `name` and `uid` substituted as necessary:
```
configSource:
configMapRef:
name: NEW_CONFIG_MAP_NAME
name: ${NEW_CONFIG_MAP_NAME}
namespace: kube-system
uid: NEW_CONFIG_MAP_UID
uid: ${NEW_CONFIG_MAP_UID}
```

### Observe that the Kubelet is using the new configuration

Once more, retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and
look for the `ConfigOK` condition in `status.conditions`. You should the message
`Using current (UID: NEW_CONFIG_MAP_UID)` when the Kubelet starts using the
look for the `KubeletConfigOK` condition in `status.conditions`. You should see the message
`using current: /api/v1/namespaces/kube-system/configmaps/${NEW_CONFIG_MAP_NAME}` when the Kubelet starts using the
new configuration.

### Deauthorize your Node fom reading the old ConfigMap
Expand Down Expand Up @@ -327,9 +327,8 @@ remove the `spec.configSource` subfield.

### Observe that the Node is using its local default configuration

After removing this subfield, you should eventually observe that the ConfigOK
condition's message reverts to either `using current (default)` or
`using current (init)`, depending on how the Node was provisioned.
After removing this subfield, you should eventually observe that the KubeletConfigOK
condition's message reverts to `using current: local`.

### Deauthorize your Node fom reading the old ConfigMap

Expand Down Expand Up @@ -366,9 +365,9 @@ Here is an example command that uses `kubectl patch`:
kubectl patch node ${NODE_NAME} -p "{\"spec\":{\"configSource\":{\"configMapRef\":{\"name\":\"${CONFIG_MAP_NAME}\",\"namespace\":\"kube-system\",\"uid\":\"${CONFIG_MAP_UID}\"}}}}"
```

## Understanding ConfigOK Conditions
## Understanding KubeletConfigOK Conditions

The following table describes several of the `ConfigOK` Node conditions you
The following table describes several of the `KubeletConfigOK` Node conditions you
might encounter in a cluster that has Dynamic Kubelet Config enabled. If you
observe a condition with `status=False`, you should check the Kubelet log for
more error details by searching for the message or reason text.
Expand All @@ -383,49 +382,33 @@ more error details by searching for the message or reason text.
<th>Status</th>
</tr>
<tr>
<td><p>using current (default)</p></td>
<td><p>current is set to the local default, and no init config was provided</p></td>
<td><p>using current: local</p></td>
<td><p>when the config source is nil, the Kubelet uses its local config</p></td>
<td><p>True</p></td>
</tr>
<tr>
<td><p>using current (init)</p></td>
<td><p>current is set to the local default, and an init config was provided</p></td>
<td><p>True</p></td>
</tr>
<tr>
<td><p>using current (UID: CURRENT_CONFIG_MAP_UID)</p></td>
<td><p>using current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</p></td>
<td><p>passing all checks</p></td>
<td><p>True</p></td>
</tr>
<tr>
<td><p>using last-known-good (default)</p></td>
<td>
<ul>
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
</ul>
</td>
<td><p>False</p></td>
</tr>
<tr>
<td><p>using last-known-good (init)</p></td>
<td><p>using last-known-good: local</p></td>
<td>
<ul>
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
</ul>
</td>
<td><p>False</p></td>
</tr>
<tr>
<td><p>using last-known-good (UID: LAST_KNOWN_GOOD_CONFIG_MAP_UID)</p></td>
<td><p>using last-known-good: /api/v1/namespaces/${LAST_KNOWN_GOOD_CONFIG_MAP_NAMESPACE}/configmaps/${LAST_KNOWN_GOOD_CONFIG_MAP_NAME}</p></td>
<td>
<ul>
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
<li>failed to load current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
<li>failed to parse current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
<li>failed to validate current: /api/v1/namespaces/${CURRENT_CONFIG_MAP_NAMESPACE}/configmaps/${CURRENT_CONFIG_MAP_NAME}</li>
</ul>
</td>
<td><p>False</p></td>
Expand All @@ -451,15 +434,15 @@ more error details by searching for the message or reason text.
<p>failed to sync, reason:</p>
<ul>
<li>failed to read Node from informer object cache</li>
<li>failed to reset to local (default or init) config</li>
<li>failed to reset to local config</li>
<li>invalid NodeConfigSource, exactly one subfield must be non-nil, but all were nil</li>
<li>invalid ObjectReference, all of UID, Name, and Namespace must be specified</li>
<li>invalid ObjectReference, UID SOME_UID does not match UID of downloaded ConfigMap SOME_OTHER_UID</li>
<li>failed to determine whether object with UID SOME_UID was already checkpointed</li>
<li>failed to download ConfigMap with name SOME_NAME from namespace SOME_NAMESPACE</li>
<li>failed to save config checkpoint for object with UID SOME_UID</li>
<li>failed to set current config checkpoint to default</li>
<li>failed to set current config checkpoint to object with UID SOME_UID</li>
<li>invalid ConfigSource.ConfigMapRef.UID: ${UID} does not match ${API_PATH}.UID: ${UID_OF_CONFIG_MAP_AT_API_PATH}</li>
<li>failed to determine whether object ${API_PATH} with UID ${UID} was already checkpointed</li>
<li>failed to download ConfigMap with name ${NAME} from namespace ${NAMESPACE}</li>
<li>failed to save config checkpoint for object ${API_PATH} with UID ${UID}</li>
<li>failed to set current config checkpoint to local config</li>
<li>failed to set current config checkpoint to object ${API_PATH} with UID ${UID}</li>
</ul>
</td>
<td><p>False</p></td>
Expand Down

0 comments on commit f5ae314

Please sign in to comment.