Wait for readiness of newly created Pods before passing them to KafkaRoller #10746
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/azp run regression |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run regression |
|
Azure Pipelines successfully started running 1 pipeline(s). |
…Roller Signed-off-by: Jakub Scholz <[email protected]>
scholzj
force-pushed
the
wait-for-readiness-of-newly-scaled-pods
branch
from
5231e16
to
f5df40c
Compare
|
/azp run regression |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run zookeeper-regression |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of change
Description
Right now, in the
KafkaReconcilerwe create/update the PodSet and then we immediately move on toKafkaRoller. That sometimes means that any newly spawned Pods are still starting and sometimes when the startup takes longer they confuse theKafkaRollerinto failing because it identifies the Pod as stuck.This PR inserts a wait between the PodSet creation/update and the KafkaRoller. That should help to avoid these issues. If for some reason the Pod does not get ready, the reconciliation will fail. But the next one will not see these Pods as new Pods anymore and will get to KafkaRoller which can fix them.
(This is a logic we already had implemented in the past when using StatefulSets.)
Checklist