Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Consider topics created by join operations internal #8520

Merged
merged 1 commit into from
Dec 17, 2021
Merged

Conversation

Gerrrr
Copy link
Contributor

@Gerrrr Gerrrr commented Dec 15, 2021

Description

Fixes #8456.

Ideally, we should probably rely on a standard predicate in KStreams, but given this issue is a release blocker, we can probably introduce this predicate to KStreams and update ksql's dependency in follow-up PRs.

Testing done

  • Added a unit test
  • Double-checked that ksqlDB's classification of internal topics matches KStreams' one.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@Gerrrr Gerrrr requested a review from a team as a code owner December 15, 2021 13:47
final String internalTopic2 = applicationId
+ "-KTABLE-FK-JOIN-SUBSCRIPTION-RESPONSE-0000000012-topic";
final String internalTopic3 = applicationId + "-something-repartition";
final String internalTopic4 = applicationId + "-someting-changelog";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: typo in something

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gerrrr Gerrrr requested review from mjsax and a team December 15, 2021 17:11
@Gerrrr Gerrrr force-pushed the fix-8456 branch 2 times, most recently from 3f3b945 to 0c9865e Compare December 15, 2021 18:24
Copy link
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gerrrr Thank you for the PR!

I have a doubt about the expression to determine the internal topics.

Comment on lines 377 to 379
&& (topicName.endsWith(KsqlConstants.STREAMS_CHANGELOG_TOPIC_SUFFIX)
|| topicName.endsWith(KsqlConstants.STREAMS_REPARTITION_TOPIC_SUFFIX));
|| topicName.endsWith(KsqlConstants.STREAMS_REPARTITION_TOPIC_SUFFIX))
|| topicName.matches(KsqlConstants.STREAMS_JOIN_REGISTRATION_TOPIC_PATTERN)
|| topicName.matches(KsqlConstants.STREAMS_JOIN_RESPONSE_TOPIC_PATTERN);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expression does not seem completely correct. Currently it is A && (B || C) || D || E. Shouldn't it be A && (B || C || D || E)? Maybe that is also the reason you get a checkstyle error that you need to suppress.
If I am right, then we are missing some tests that check the negative case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I fixed it and added checks for the negative cases in 9e2652e.

Copy link
Member

@cadonna cadonna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Gerrrr
Copy link
Contributor Author

Gerrrr commented Dec 17, 2021

Even though CI is broken, I think that it is ok to merge it because:

  • There was a green build (and other builds with unrelated failures) before the CI broke.
  • The change itself is very much isolated and likely won't affect other tests.
  • I tested the change locally.
  • The issue is a release blocker.

@Gerrrr Gerrrr merged commit ebe0c49 into master Dec 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Two internal topics are not cleaned up after terminating query for FK join
3 participants