Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

syncer(dm): save table checkpoint after a DDL is filtered #5273

Merged
merged 4 commits into from
Apr 27, 2022
Merged

syncer(dm): save table checkpoint after a DDL is filtered #5273

merged 4 commits into from
Apr 27, 2022

Conversation

lance6716
Copy link
Contributor

@lance6716 lance6716 commented Apr 26, 2022 •
edited
Loading

Signed-off-by: lance6716 [email protected]

What problem does this PR solve?

Issue Number: ref #5272

What is changed and how it works?

as title

Check List

Tests

  • Integration test

Related changes

  • Need to cherry-pick to the release branch

Release note 

save table checkpoint after a DDL is filtered

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Apr 26, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • D3Hunter
  • GMHDBJD

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. labels Apr 26, 2022
@lance6716 lance6716 added the area/dm Issues or PRs related to DM. label Apr 26, 2022
@ti-chi-bot ti-chi-bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Apr 26, 2022
@lance6716
Copy link
Contributor Author

/hold

test is invalid

@ti-chi-bot ti-chi-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 26, 2022
@lance6716
fix test
280c14a 
Signed-off-by: lance6716 <[email protected]>
@ti-chi-bot ti-chi-bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Apr 26, 2022
@lance6716
Copy link
Contributor Author

/run-dm-integration-test
/unhold

@ti-chi-bot ti-chi-bot added needs-cherry-pick-release-5.3 Should cherry pick this PR to release-5.3 branch. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Apr 26, 2022
GMHDBJD
GMHDBJD approved these changes Apr 26, 2022
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dm/tests/tracker_ignored_ddl/run.sh
Comment on lines +49 to 50
sleep 3
run_dm_ctl_with_retry $WORK_DIR "127.0.0.1:$MASTER_PORT" \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems the run_dm_ctl_with_retry already meets the requirements

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to wait some time to see if an error is going to happen

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Apr 26, 2022
D3Hunter
D3Hunter reviewed Apr 26, 2022
View reviewed changes
dm/syncer/checkpoint.go
@@ -128,6 +128,7 @@ func (b *binlogPoint) rollback(schemaTracker *schema.Tracker, schema string) (is
b.flushedPoint.location.ResetSuffix()
b.savedPoint.location = b.flushedPoint.location
if b.savedPoint.ti == nil {
// TODO: if we forget to save table info for table checkpoint, this is also nil!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

table point which rollbacks before can also be nil

dm/syncer/syncer.go Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 27, 2022
@lance6716
Copy link
Contributor Author

/merge
/run-all-tests

@codecov-commenter
Copy link

Codecov Report

Merging #5273 (b37cfac) into master (687e248) will increase coverage by 0.0145%.
The diff coverage is 53.8933%.

Flag Coverage Δ
cdc 60.9608% <53.8933%> (+0.3621%) ⬆️
dm 52.3346% <ø> (-0.1350%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master      #5273        +/-   ##
================================================
+ Coverage   56.1240%   56.1385%   +0.0145%     
================================================
  Files           522        524         +2     
  Lines         65325      68730      +3405     
================================================
+ Hits          36663      38584      +1921     
- Misses        25094      26412      +1318     
- Partials       3568       3734       +166

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: b37cfac

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 27, 2022
@lance6716
Copy link
Contributor Author

/run-dm-integration-test

@ti-chi-bot ti-chi-bot merged commit 7744c05 into pingcap:master Apr 27, 2022
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 27, 2022
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5290.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Apr 27, 2022
@lance6716 @ti-chi-bot
This is an automated cherry-pick of pingcap#5273
397bcc6 
Signed-off-by: ti-chi-bot <[email protected]>
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Apr 27, 2022
@lance6716 @ti-chi-bot
This is an automated cherry-pick of pingcap#5273
040c491 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 27, 2022
Closed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5291.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request Apr 27, 2022
@lance6716 @ti-chi-bot
This is an automated cherry-pick of pingcap#5273
06592b9 
Signed-off-by: ti-chi-bot <[email protected]>
@ti-chi-bot ti-chi-bot mentioned this pull request Apr 27, 2022
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5292.

ti-chi-bot added a commit that referenced this pull request Apr 27, 2022
@ti-chi-bot
syncer(dm): save table checkpoint after a DDL is filtered (#5273) (#5290
f393119 
)

ref #5272
@lance6716 lance6716 mentioned this pull request May 12, 2022
Closed
ti-chi-bot added a commit that referenced this pull request May 25, 2022
@ti-chi-bot
syncer(dm): save table checkpoint after a DDL is filtered (#5273) (#5292
f934ba6 
)

ref #5272
@lance6716 lance6716 deleted the fix-5272 branch October 13, 2022 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@D3Hunter D3Hunter D3Hunter approved these changes

@GMHDBJD GMHDBJD GMHDBJD approved these changes

Assignees
No one assigned
Labels
area/dm Issues or PRs related to DM. needs-cherry-pick-release-5.3 Should cherry pick this PR to release-5.3 branch. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lance6716 @ti-chi-bot @codecov-commenter @D3Hunter @GMHDBJD