Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syncer(dm): fix failed row skipped due to incorrect checkpoint flush #5295

Merged
merged 2 commits into from
Apr 27, 2022

Conversation

D3Hunter
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #5279

What is changed and how it works?

  • checkpoint flush should wait all dml before it finished.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update key monitor metrics in both TiCDC document and official document

Release note

`None`.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 27, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • lance6716

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. labels Apr 27, 2022
@D3Hunter D3Hunter added the area/dm Issues or PRs related to DM. label Apr 27, 2022
@ti-chi-bot ti-chi-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 27, 2022
@D3Hunter D3Hunter added needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. and removed needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. labels Apr 27, 2022
@D3Hunter
Copy link
Contributor Author

/cc @lance6716 @GMHDBJD

Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 27, 2022
@lance6716
Copy link
Contributor

/run-dm-integration-test

Copy link
Contributor

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe my old thought is don't let handleJob calls handleJob itself, but I forget to flush the jobs in queue 😂

I'm afraid handleJob calling handleJob itself will cause stack overflow in some bad cases, but flushIfOutdated is a common logic so it can also be put inside handleJob. No idea 🤔

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 27, 2022
@D3Hunter
Copy link
Contributor Author

/run-dm-compatibility-test
/run-integration-test
/run-kafka-integration-test

@D3Hunter
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 67c744c

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 27, 2022
@ti-chi-bot ti-chi-bot merged commit cd90321 into pingcap:master Apr 27, 2022
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Apr 27, 2022
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5297.

@D3Hunter D3Hunter deleted the fix-skipped-row branch April 27, 2022 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/dm Issues or PRs related to DM. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

failed rows are skipped due to checkpoint flush
4 participants