Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update repartition processor names for KAFKA-9098 #3802

Merged

Conversation

bbejeck
Copy link
Contributor

@bbejeck bbejeck commented Nov 8, 2019

Description

When Kafka Streams introduced the ability to name repartition topics, the accompanying Filter, Source and Sink processors continued to use the generated names. With the addition of KIP-307, users can now name every operator in a Streams DSL application. The repartition Filter, Source, and Sink operators are internally generated and the users have no ability to name these operations.

So if a user goes through and names every operation in their DSL application, when viewing a topology description they will still see some KSTREAM-XXX-00000000N names (if there is a repartition topic), which could be confusing.

Now when users provide a name for the repartition topic Kafka Streams will name the processors for the repartition in the following manner:

repartition topic name = my-aggregation-repartition
KSTREAM-FILTER-00000000N -> my-aggregation-repartition-filter
KSTREAM-SINK-00000000N -> my-aggregation-repartition-sink
KSTREAM-SOURCE-00000000N -> my-aggregation-repartition-source

Since these processors are stateless this is a non-breaking change

This PR should only get merged after the AK PR is merged to CCS.

Testing done

I ran mvn clean package against the branch with the Kafka Streams changes with updated expected topology files and tests for all modules passed.

Reviewer checklist

  • Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
  • Ensure relevant issues are linked (description should include text like "Fixes #")

@bbejeck bbejeck requested a review from a team as a code owner November 8, 2019 18:00
Copy link
Contributor

@rodesai rodesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rodesai rodesai merged commit 2b86cd8 into master Nov 9, 2019
@bbejeck bbejeck deleted the fix_repartition_processor_name_changes_for_KAFKA-9098 branch November 11, 2019 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants