-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[controller][compat] Controller part change for supporting separate real-time topic functionality for hybrid stores. #1172
Conversation
670f703
to
42639a4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Hao for the change. I take one pass, overall it looks good, I left a few small comments for clarification.
internal/venice-common/src/main/java/com/linkedin/venice/meta/Version.java
Outdated
Show resolved
Hide resolved
...s/venice-controller/src/main/java/com/linkedin/venice/controller/VeniceParentHelixAdmin.java
Show resolved
Hide resolved
...ce-controller/src/main/java/com/linkedin/venice/ingestion/control/RealTimeTopicSwitcher.java
Outdated
Show resolved
Hide resolved
2afaf9e
to
69426a9
Compare
69426a9
to
acfd2c5
Compare
internal/venice-common/src/main/java/com/linkedin/venice/ConfigKeys.java
Show resolved
Hide resolved
f7b98bb
to
b47f726
Compare
09ea3a0
to
8449734
Compare
8449734
to
354c78e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the change! Let's get the unrelated flaky test passed (should address that separately outside this PR scope) and we can get this merged.
Summary, imperative, start upper case, don't end with a period
This change add store version level config
separateRealTimeTopicEnabled
to allow creating separate real-time topic for isolate bulk real-time traffic in short time range from incremental push. There is cluster level config for enabling this config for all newly converted hybrid store with incremental push enabled.separateRealTimeTopicEnabled
is enabled when hybrid enabled store version is added and normal real-time topic is created.Added Integration test to see if incremental push job would send expected traffic to the separate real-time topic after new store is turned this feature.
PS: skip the flaky test:
testDaVinciMemoryLimitShouldFailLargeDataPushAndResumeHybridStore
.How was this PR tested?
Does this PR introduce any user-facing changes?