Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TiCDC changefeed sequential initialization #3588

Closed
amyangfei opened this issue Nov 23, 2021 · 3 comments
Closed

TiCDC changefeed sequential initialization #3588

amyangfei opened this issue Nov 23, 2021 · 3 comments
Assignees
@3AceShowHand
Labels
area/ticdc Issues or PRs related to TiCDC. subject/new-feature Denotes an issue or pull request adding a new feature.

Comments

@amyangfei
Copy link
Contributor

amyangfei commented Nov 23, 2021 •
edited
Loading

Is your feature request related to a problem?

Supposing a TiCDC cluster has multiple changefeeds, once TiCDC owner is transferred, the new owner will initialize changefeed in sequence. The initialization of a changefeed could cost long time (such as connecting and verifying with Kafka sink), which will lead to large checkpoint lag.

Describe the feature you'd like

  • Make changefeed initialization asynchronous.
  • or make part of the changefeed initialization phase asynchronous, such as the new sink phase.

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

No response
@amyangfei amyangfei added the subject/new-feature Denotes an issue or pull request adding a new feature. label Nov 23, 2021
@amyangfei
Copy link
Contributor Author

amyangfei commented Nov 23, 2021 •
edited
Loading

We have met owner stuck issue in https://github.com/pingcap/ticdc/issues/3352 because the initialization of a sarama.(*Broker) blocks the owner goroutine from doing everything else.

@asddongmen asddongmen added the area/ticdc Issues or PRs related to TiCDC. label Nov 25, 2021
@amyangfei
Copy link
Contributor Author

amyangfei commented Dec 15, 2021 •
edited
Loading

New feature will be deliveried in https://github.com/pingcap/ticdc/pull/3598

@amyangfei amyangfei mentioned this issue Dec 15, 2021
Closed
@amyangfei
Copy link
Contributor Author

Closed by #3598

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@3AceShowHand 3AceShowHand

Labels
area/ticdc Issues or PRs related to TiCDC. subject/new-feature Denotes an issue or pull request adding a new feature.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@amyangfei @3AceShowHand @asddongmen