[system test] Auto-rebalance feature test case #10666

see-quick · 2024-10-02T11:41:34Z

Type of change

Enhancement / new feature
Documentation

Description

This PR adds a system test to cover the basic functionality of the auto-balance feature.

Checklist

Write tests
Make sure all tests pass

development-docs/systemtests/io.strimzi.systemtest.cruisecontrol.CruiseControlST.md

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlST.java

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java

im-konge

Few comments, thanks :)

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaRebalanceUtils.java

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java

see-quick · 2024-10-03T10:04:16Z

/azp run regression

azure-pipelines · 2024-10-03T10:04:26Z

Azure Pipelines successfully started running 1 pipeline(s).

development-docs/systemtests/io.strimzi.systemtest.cruisecontrol.CruiseControlST.md

ppatierno · 2024-10-03T14:52:26Z

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlST.java

+            @Step(value = "Check that topic replicas are moved to the new brokers.", expected = "Topic replicas are distributed onto the newly added brokers."),
+            @Step(value = "Scale Kafka down to the original number of brokers.", expected = "Kafka brokers are scaled down, and Cruise Control initiates rebalancing in REMOVE_BROKERS mode."),
+            @Step(value = "Verify that Kafka auto-rebalance status transitions to RebalanceOnScaleDown and then back to Idle.", expected = "Auto-rebalance status moves to RebalanceOnScaleDown during scaling down and returns to Idle after rebalancing completes."),
+            @Step(value = "Confirm that the cluster is stable after scaling operations.", expected = "Cluster returns to a stable state with initial number of brokers and Cruise Control completed the rebalancing.")


@see-quick see the comments I made on the description, they apply here as well.

see-quick · 2024-10-04T09:47:17Z

Converting to the draft I have faced issue [1] multiple times (3/5 times race conditions happen).

[1] - #10631

Signed-off-by: see-quick <[email protected]>

see-quick · 2024-10-17T09:47:23Z

/azp run regression

azure-pipelines · 2024-10-17T09:47:35Z

Azure Pipelines successfully started running 1 pipeline(s).

ppatierno · 2024-10-17T16:06:54Z

@see-quick build and regression are green, I guess we can move this back as ready to be reviewed.

ppatierno

LGTM

ppatierno · 2024-10-18T09:23:30Z

@im-konge can you have another pass on this please?

Signed-off-by: see-quick <[email protected]>

see-quick · 2024-10-21T08:52:44Z

/azp run regression

azure-pipelines · 2024-10-21T08:52:55Z

Azure Pipelines successfully started running 1 pipeline(s).

im-konge

Just two small nits in the logging stuff. Thanks

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java

Signed-off-by: see-quick <[email protected]>

see-quick · 2024-10-21T14:38:30Z

/azp run regression

azure-pipelines · 2024-10-21T14:38:41Z

Azure Pipelines successfully started running 1 pipeline(s).

scholzj · 2024-10-21T15:38:16Z

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlST.java

+        resourceManager.createResourceWithoutWait(
+            KafkaRebalanceTemplates.kafkaRebalance(testStorage.getNamespaceName(), testStorage.getClusterName())
+                .editMetadata()
+                    .addToAnnotations(Annotations.ANNO_STRIMZI_IO_REBALANCE, "template")


So, I guess this needs to be fixed to the new format?

@see-quick it's weird it's still this way. We had a chat to update it to the new strimzi.io/rebalance-template: true and it was also done, or? I was pretty sure about that, for this reason approved at some point.

Yeah, but we speak offline with Paolo. He will rebase :)

scholzj · 2024-10-21T15:38:21Z

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlST.java

+                .build(),
+            KafkaRebalanceTemplates.kafkaRebalance(testStorage.getNamespaceName(), testStorage.getClusterName())
+                .editMetadata()
+                    .addToAnnotations(Annotations.ANNO_STRIMZI_IO_REBALANCE, "template")


So, I guess this needs to be fixed to the new format?

Yes, ditto as above.

scholzj · 2024-10-21T15:39:17Z

systemtest/src/test/java/io/strimzi/systemtest/cruisecontrol/CruiseControlST.java

+                                .withMode(KafkaRebalanceMode.ADD_BROKERS)
+                                .withNewTemplate(scaleUpKafkaRebalanceTemplateName)
+                                .build(),
+                            new KafkaAutoRebalanceConfigurationBuilder()
+                                .withMode(KafkaRebalanceMode.REMOVE_BROKERS)


This will be likely broken by #10744?

Yes it will be. At this point I think it's better waiting for #10744 to be merged (regression running) and then rebasing this one or I have to fix it in my PR when this is merged.

Signed-off-by: see-quick <[email protected]>

see-quick added needs review System tests labels Oct 2, 2024

see-quick requested review from ppatierno and a team October 2, 2024 11:41

see-quick self-assigned this Oct 2, 2024

ppatierno reviewed Oct 2, 2024

View reviewed changes

see-quick force-pushed the auto-rebalance-scale-up-down branch from 73c260d to 8325741 Compare October 2, 2024 15:11

ppatierno reviewed Oct 2, 2024

View reviewed changes

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java Outdated Show resolved Hide resolved

im-konge reviewed Oct 3, 2024

View reviewed changes

see-quick added this to the 0.44.0 milestone Oct 3, 2024

ppatierno reviewed Oct 3, 2024

View reviewed changes

see-quick marked this pull request as draft October 4, 2024 09:47

see-quick force-pushed the auto-rebalance-scale-up-down branch from dc81cd2 to d66e6a6 Compare October 10, 2024 09:51

ppatierno mentioned this pull request Oct 15, 2024

Fixed stuck reconciliation on failed request rebalance #10717

Merged

4 tasks

see-quick added 7 commits October 17, 2024 11:33

for Paolo

b7eb09a

Signed-off-by: see-quick <[email protected]>

add javadoc to aux methods

b940695

Signed-off-by: see-quick <[email protected]>

[system test] Auto-rebalance feature test case

85b0da7

Signed-off-by: see-quick <[email protected]>

update images

a25bf34

Signed-off-by: see-quick <[email protected]>

update Paolo review

1d12c3b

Signed-off-by: see-quick <[email protected]>

update Paolo and Lukas reviw

d11acc8

Signed-off-by: see-quick <[email protected]>

a few update to docs

5ccc96a

Signed-off-by: see-quick <[email protected]>

see-quick force-pushed the auto-rebalance-scale-up-down branch from d66e6a6 to 5ccc96a Compare October 17, 2024 09:33

see-quick marked this pull request as ready for review October 17, 2024 18:50

ppatierno approved these changes Oct 18, 2024

View reviewed changes

remove TODO

d225b72

Signed-off-by: see-quick <[email protected]>

im-konge approved these changes Oct 21, 2024

View reviewed changes

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java Outdated Show resolved Hide resolved

systemtest/src/main/java/io/strimzi/systemtest/utils/kafkaUtils/KafkaUtils.java Outdated Show resolved Hide resolved

see-quick removed the needs review label Oct 21, 2024

Lukas review

eb6695f

Signed-off-by: see-quick <[email protected]>

see-quick force-pushed the auto-rebalance-scale-up-down branch from 8e8e0f0 to eb6695f Compare October 21, 2024 14:36

scholzj reviewed Oct 21, 2024

View reviewed changes

update templates

1b8f160

Signed-off-by: see-quick <[email protected]>

see-quick merged commit ff8521a into strimzi:main Oct 22, 2024
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[system test] Auto-rebalance feature test case #10666

[system test] Auto-rebalance feature test case #10666

see-quick commented Oct 2, 2024 •

edited

Loading

im-konge left a comment

see-quick commented Oct 3, 2024

azure-pipelines bot commented Oct 3, 2024

ppatierno Oct 3, 2024

see-quick commented Oct 4, 2024 •

edited

Loading

see-quick commented Oct 17, 2024

azure-pipelines bot commented Oct 17, 2024

ppatierno commented Oct 17, 2024

ppatierno left a comment

ppatierno commented Oct 18, 2024

see-quick commented Oct 21, 2024

azure-pipelines bot commented Oct 21, 2024

im-konge left a comment

see-quick commented Oct 21, 2024

azure-pipelines bot commented Oct 21, 2024

scholzj Oct 21, 2024

ppatierno Oct 21, 2024 •

edited

Loading

see-quick Oct 21, 2024

scholzj Oct 21, 2024

ppatierno Oct 21, 2024

scholzj Oct 21, 2024

ppatierno Oct 21, 2024 •

edited

Loading

[system test] Auto-rebalance feature test case #10666

[system test] Auto-rebalance feature test case #10666

Conversation

see-quick commented Oct 2, 2024 • edited Loading

Type of change

Description

Checklist

im-konge left a comment

Choose a reason for hiding this comment

see-quick commented Oct 3, 2024

azure-pipelines bot commented Oct 3, 2024

ppatierno Oct 3, 2024

Choose a reason for hiding this comment

see-quick commented Oct 4, 2024 • edited Loading

see-quick commented Oct 17, 2024

azure-pipelines bot commented Oct 17, 2024

ppatierno commented Oct 17, 2024

ppatierno left a comment

Choose a reason for hiding this comment

ppatierno commented Oct 18, 2024

see-quick commented Oct 21, 2024

azure-pipelines bot commented Oct 21, 2024

im-konge left a comment

Choose a reason for hiding this comment

see-quick commented Oct 21, 2024

azure-pipelines bot commented Oct 21, 2024

scholzj Oct 21, 2024

Choose a reason for hiding this comment

ppatierno Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

see-quick Oct 21, 2024

Choose a reason for hiding this comment

scholzj Oct 21, 2024

Choose a reason for hiding this comment

ppatierno Oct 21, 2024

Choose a reason for hiding this comment

scholzj Oct 21, 2024

Choose a reason for hiding this comment

ppatierno Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

see-quick commented Oct 2, 2024 •

edited

Loading

see-quick commented Oct 4, 2024 •

edited

Loading

ppatierno Oct 21, 2024 •

edited

Loading

ppatierno Oct 21, 2024 •

edited

Loading