Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve QoS when GMS writes to MCL #6377

Closed
djordje-mijatovic opened this issue Nov 7, 2022 · 1 comment · Fixed by #6381
Closed

Improve QoS when GMS writes to MCL #6377

djordje-mijatovic opened this issue Nov 7, 2022 · 1 comment · Fixed by #6381
Labels

Comments

@djordje-mijatovic
Copy link
Contributor

GMS writes to a Kafka topic to report all data changes. The topic is treated as a change log but GMS is very optimistic when writing to it. We'd like a strong guarantee that any data change accepted by GMS will be written to Kafka.
There is currently a Callback mechanism in place but it is used only for logging purpose - we would like to extend mentioned Callback function to retry send operation for few time (configurable) and after few fails log the error as it is now.
Pay attention that failed on one topic should not have effect (slow down) on sending to other topics.

@github-actions
Copy link

github-actions bot commented Dec 8, 2022

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant