[DRAFT] docs: update do-not-disrupt description #6977
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for karpenter-docs-prod ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Pull Request Test Coverage Report for Build 10802199988Details
💛 - Coveralls |
| @@ -411,6 +411,7 @@ Karpenter should now be pulling and operating against the v1beta1 APIVersion as | |||
| * API Rename: NodePool’s ConsolidationPolicy `WhenUnderutilized` is now renamed to `WhenEmptyOrUnderutilized` | |||
| * Behavior Changes: | |||
| * Expiration is now forceful and begins draining as soon as it’s expired. Karpenter does not wait for replacement capacity to be available before draining, but will start provisioning a replacement as soon as the node is expired and begins draining. | |||
| * Pods with the `karpenter.sh/do-not-disrupt` annotation now block node termination. Termination of a node with these pods will be blocked until those pods are removed, enter a terminating or terminal state, or the NodeClaims's TerminationGracePeriod has expired. | |||
There was a problem hiding this comment.
Can we link out to the section of the docs that describes TerminationGracePeriod here?
| @@ -274,8 +282,14 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa | |||
|
|
|||
| ### Pod-Level Controls | |||
|
|
|||
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. This is useful for pods that you want to run from start to finish without disruption. By opting pods out of this disruption, you are telling Karpenter that it should not voluntarily remove a node containing this pod. | |||
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. | |||
| You can treat this annotation as a single-node, permanently blocking PDB. | |||
There was a problem hiding this comment.
Suggested change
| You can treat this annotation as a single-node, permanently blocking PDB. | |
| You can treat this annotation as a single-pod, permanently blocking PDB. |
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. | ||
| You can treat this annotation as a single-node, permanently blocking PDB. | ||
| This has the following consequences: | ||
| - Nodes with `do-not-disrupt` pods will be excluded from **voluntary** disruption, i.e. [Consolidation]({{<ref "#consolidation" >}}) and [Drift]({{<ref "#drift" >}}). |
There was a problem hiding this comment.
Doesn't voluntary disruption include drift? I'm not sure that they are considered different things here
| You can treat this annotation as a single-node, permanently blocking PDB. | ||
| This has the following consequences: | ||
| - Nodes with `do-not-disrupt` pods will be excluded from **voluntary** disruption, i.e. [Consolidation]({{<ref "#consolidation" >}}) and [Drift]({{<ref "#drift" >}}). | ||
| - Like pods with a blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). |
There was a problem hiding this comment.
Suggested change
| - Like pods with a blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). | |
| - Like pods with a fully blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). |
| You can treat this annotation as a single-node, permanently blocking PDB. | ||
| This has the following consequences: | ||
| - Nodes with `do-not-disrupt` pods will be excluded from **voluntary** disruption, i.e. [Consolidation]({{<ref "#consolidation" >}}) and [Drift]({{<ref "#drift" >}}). | ||
| - Like pods with a blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). |
There was a problem hiding this comment.
Suggested change
| - Like pods with a blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). | |
| - Like pods with a blocking PDB, pods with the `do-not-disrupt` annotation will **not** be gracefully evicted by the [Termination Controller]({{ref "#terminationcontroller"}}). These pods will either run to completion or be forcefully terminated when the node is near its terminationGracePeriod |
Consider linking to terminationGracePeriod if you update the docs wording in this way too
| @@ -274,8 +282,14 @@ Duration and Schedule must be defined together. When omitted, the budget is alwa | |||
|
|
|||
| ### Pod-Level Controls | |||
|
|
|||
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. This is useful for pods that you want to run from start to finish without disruption. By opting pods out of this disruption, you are telling Karpenter that it should not voluntarily remove a node containing this pod. | |||
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. | |||
There was a problem hiding this comment.
Suggested change
| You can block Karpenter from voluntarily choosing to disrupt certain pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. | |
| You can block Karpenter from voluntarily disrupting pods by setting the `karpenter.sh/do-not-disrupt: "true"` annotation on the pod. |
|
|
||
| You can set a NodePool's `terminationGracePeriod` through the `spec.template.spec.terminationGracePeriod` field. This field defines the duration of time that a node can be draining before it's forcibly deleted. A node begins draining when it's deleted. Pods will be deleted preemptively based on its TerminationGracePeriodSeconds before this terminationGracePeriod ends to give as much time to cleanup as possible. Note that if your pod's terminationGracePeriodSeconds is larger than this terminationGracePeriod, Karpenter may forcibly delete the pod before it has its full terminationGracePeriod to cleanup. | ||
| You can set a NodePool's `terminationGracePeriod` through the `spec.template.spec.terminationGracePeriod` field. | ||
| This field defines the duration of time that a node can be draining before it's forcibly deleted. |
There was a problem hiding this comment.
Suggested change
| This field defines the duration of time that a node can be draining before it's forcibly deleted. | |
| This field defines the duration of time that a node can be draining before it's forcibly deleted. |
|
|
||
| This is especially useful in combination with `nodepool.spec.template.spec.expireAfter` to define an absolute maximum on the lifetime of a node, where a node is deleted at `expireAfter` and finishes draining within the `terminationGracePeriod` thereafter. Pods blocking eviction like PDBs and do-not-disrupt will block full draining until the `terminationGracePeriod` is reached. | ||
| This is especially useful in combination with `nodepool.spec.template.spec.expireAfter` to define an absolute maximum on the lifetime of a node, where a node is deleted at `expireAfter` and finishes draining within the `terminationGracePeriod` thereafter. | ||
| Pods blocking eviction like PDBs and do-not-disrupt will block full draining until the `terminationGracePeriod` is reached. |
There was a problem hiding this comment.
Suggested change
| Pods blocking eviction like PDBs and do-not-disrupt will block full draining until the `terminationGracePeriod` is reached. | |
| Pods blocking eviction like PDBs and `do-not-disrupt` will block full draining until the `terminationGracePeriod` is reached. |
|
|
||
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining after it's lived for `23h`. Let's say a `do-not-disrupt` pod has `TerminationGracePeriodSeconds` set to `300` seconds. If the node hasn't been fully drained after `55m`, Karpenter will delete the pod to allow it's full `terminationGracePeriodSeconds` to cleanup. If no pods are blocking draining, Karpenter will cleanup the node as soon as the node is fully drained, rather than waiting for the NodeClaim's `terminationGracePeriod` to finish. | ||
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining after it's lived for `23h`. |
There was a problem hiding this comment.
Suggested change
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining after it's lived for `23h`. | |
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining `23h` after its creation. The NodeClaim will then be allowed to drain for up to `1h` before its forcefully terminated from the cluster. |
|
|
||
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining after it's lived for `23h`. Let's say a `do-not-disrupt` pod has `TerminationGracePeriodSeconds` set to `300` seconds. If the node hasn't been fully drained after `55m`, Karpenter will delete the pod to allow it's full `terminationGracePeriodSeconds` to cleanup. If no pods are blocking draining, Karpenter will cleanup the node as soon as the node is fully drained, rather than waiting for the NodeClaim's `terminationGracePeriod` to finish. | ||
| For instance, a NodeClaim with `terminationGracePeriod` set to `1h` and an `expireAfter` set to `23h` will begin draining after it's lived for `23h`. | ||
| Let's say a `do-not-disrupt` pod has `TerminationGracePeriodSeconds` set to `300` seconds. |
There was a problem hiding this comment.
Suggested change
| Let's say a `do-not-disrupt` pod has `TerminationGracePeriodSeconds` set to `300` seconds. | |
| Let's say a `do-not-disrupt` pod has `TerminationGracePeriodSeconds` set to `300` seconds (`5m`). |
jmdeal
changed the title
|
This PR has been inactive for 14 days. StaleBot will close this stale PR after 14 more days of inactivity. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #N/A
Description
Updates the description for
karpenter.sh/do-not-disruptto reflect the changes made when TGP was introduced.How was this change tested?
Does this change impact docs?
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.