Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Fix and improve the test QuickDegradeAndRestoreCommandTopicIntegrationTest #8275

Merged
merged 3 commits into from
Oct 22, 2021
Merged

test: Fix and improve the test QuickDegradeAndRestoreCommandTopicIntegrationTest #8275

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3454a96
test: Improve the test QuickDegradeAndRestoreCommandTopicIntegrationT…
mkandaswamy Oct 21, 2021
4afc2b9
Fix a race condition in QuickDegradeAndRestoreCommandTopicIntegration…
mkandaswamy Oct 21, 2021
c2f61a3
Restore the default to true for KafkaConfig: auto.create.topics.enabl…
mkandaswamy Oct 21, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .../io/confluent/ksql/api/integration/QuickDegradeAndRestoreCommandTopicIntegrationTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,12 +117,13 @@ public void shouldBeInDegradeModeAfterCmdTopicDeleteAndRestart() throws Excepti
// When
// Delete the command topic and restart
TEST_HARNESS.deleteTopics(Collections.singletonList(commandTopic));
assertThatEventually("Topic Deleted", this::isCommandTopicDeleted, is(true));
REST_APP.stop();
REST_APP.start();

// Then
assertThatEventually("Topic Deleted", this::isCommandTopicDeleted, is(true));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good check

assertThatEventually("Degraded State", this::isDegradedState, is(true));
assertThat(TEST_HARNESS.topicExists(commandTopic), is(false));
REST_APP.stop();
KsqlRestoreCommandTopic.main(
new String[]{
Expand All @@ -139,6 +140,10 @@ public void shouldBeInDegradeModeAfterCmdTopicDeleteAndRestart() throws Excepti
assertThatEventually("Degraded State", this::isDegradedState, is(false));
}

private boolean isCommandTopicDeleted() {
return !TEST_HARNESS.topicExists(commandTopic);
}

private boolean isDegradedState() {
// If in degraded state, then the following command will return a warning
final List<KsqlEntity> response = makeKsqlRequest(
Expand Down
5 changes: 3 additions & 2 deletions ...b-test-util/src/main/java/io/confluent/ksql/test/util/EmbeddedSingleNodeKafkaCluster.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -576,8 +576,9 @@ private Properties buildBrokerConfig(final String logDir) {
// Need to know where ZK is:
config.put(KafkaConfig.ZkConnectProp(), zookeeper.connectString());
config.put(AclAuthorizer.ZkUrlProp(), zookeeper.connectString());
// Do not require tests to explicitly create tests:
config.put(KafkaConfig.AutoCreateTopicsEnableProp(), true);
// Create topics explicitly when needed to avoid a race which
// automatically recreates deleted command topic:
config.put(KafkaConfig.AutoCreateTopicsEnableProp(), false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried that there may be many tests that rely on automatically creating topics. Is there way to set this config but at the test level instead?

Copy link
Member

@nateab nateab Oct 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at ApiIntegrationTest, when it creates the TEST_HARNESS it configure the properties it needs for the kafka cluster as follows:

private static final IntegrationTestHarness TEST_HARNESS = IntegrationTestHarness.builder()
      
      .withKafkaCluster(

          EmbeddedSingleNodeKafkaCluster.newBuilder()

              .withoutPlainListeners()

              .withSaslSslListeners()
      ).build();

Can we do something similar here and create a function to remove the auto topic creation for just this test?

Copy link
Member Author

@mkandaswamy mkandaswamy Oct 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, seem like few tests do depend on that setting to be true. I've addressed this in my latest commit. For now, only QuickDegradeAndRestoreCommandTopicIntegrationTest will set automatically creating topics to be false.

// Default to small number of partitions for auto-created topics:
config.put(KafkaConfig.NumPartitionsProp(), 1);
// Allow tests to delete topics:
Expand Down