Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ticdc/owner: Fix ddl special comment syntax error (#3845) #3982

Closed
wants to merge 0 commits into from
Closed

ticdc/owner: Fix ddl special comment syntax error (#3845) #3982

wants to merge 0 commits into from

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #3845

What problem does this PR solve?

close https://github.com/pingcap/ticdc/issues/3755

What is changed and how it works?

not use regex to parse and add special TiDB comments, If a DDL already has special comment like /*!90000 SHARD_ROW_ID_BITS=4 PRE_SPLIT_REGIONS=3 */, it wil translate that DDL to /*!90000 /*T! SHARD_ROW_ID_BITS=4 PRE_SPLIT_REGIONS=3 */ */, TiDB will throw a syntax error.
this PR use the TiDB SQL parser to parse the SQL and the Restore it with format.RestoreStringSingleQuotes|format.RestoreNameBackQuotes|format.RestoreKeyWordUppercase|format.RestoreTiDBSpecialComment flag

Check List

Tests

  • Unit test

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to update key monitor metrics in both TiCDC document and official document

Release note 

Fix syntax error if DDL has a special comment.

@ti-chi-bot ti-chi-bot mentioned this pull request Dec 20, 2021
Merged
@ti-chi-bot
Copy link
Member Author

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/cherry-pick-not-approved component/ddl DDL component. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-5.1 This PR is cherry-picked to release-5.1 from a source PR. labels Dec 20, 2021
@ti-chi-bot ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Dec 20, 2021
@overvenus overvenus added this to the v5.1.4 milestone Jan 6, 2022
@VelocityLight VelocityLight added the cherry-pick-approved Cherry pick PR approved by release team. label Jan 19, 2022
@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-all-tests

@ti-chi-bot ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 19, 2022
@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-all-tests

overvenus
overvenus reviewed Jan 19, 2022
View reviewed changes
go.mod Outdated
github.com/pingcap/parser v0.0.0-20210427084954-8e8ed7927bde
github.com/pingcap/tidb v1.1.0-beta.0.20210508083641-8ed1d9d4a798
github.com/pingcap/tidb-tools v4.0.9-0.20201127090955-2707c97b3853+incompatible
github.com/pingcap/tidb/parser v0.0.0-20220119053344-a4e8ed59bdfd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not keep using pingcap/parser above at line 46?

@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 19, 2022
@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-all-tests

1 similar comment
@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-all-tests

@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-integration-tests
/run-kafka-integration-test

@codecov-commenter
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (release-5.1@0cf9c95). Click here to learn what that means.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release-5.1      #3982   +/-   ##
================================================
  Coverage               ?   62.1569%           
================================================
  Files                  ?        161           
  Lines                  ?      16978           
  Branches               ?          0           
================================================
  Hits                   ?      10553           
  Misses                 ?       5519           
  Partials               ?        906

@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-all-tests

@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-kafka-integration-test

overvenus
overvenus reviewed Jan 19, 2022
View reviewed changes
go.mod Outdated
golang.org/x/time v0.0.0-20191024005414-555d28b269f0
google.golang.org/grpc v1.27.1
google.golang.org/grpc v1.29.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
google.golang.org/grpc v1.29.1
google.golang.org/grpc v1.27.1

Could you keep 1.27.1?

proto/EntryProtocol.proto Outdated
@@ -210,7 +210,7 @@ enum EntryType{
ROWDATA = 2;
TRANSACTIONEND = 3;
/** 心跳类型，内部使用，外部暂不可见，可忽略 **/
HEARTBEAT = 4;
ENTRYHEARTBEAT = 4;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ENTRYHEARTBEAT = 4;
HEARTBEAT = 4;

It looks like a breaking change for users.

tests/integration_tests/resolve_lock/main.go Outdated
Comment on lines 39 to 41
"github.com/tikv/client-go/v2/oracle"
"github.com/tikv/client-go/v2/tikv"
"github.com/tikv/client-go/v2/tikvrpc"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you keep use tidb/store/tikv/...? TiDB release-5.1 has not switched to tikv/client-go yet.

@sdojjy sdojjy force-pushed the cherry-pick-3845-to-release-5.1 branch 2 times, most recently from 0775b69 to c1661df Compare January 19, 2022 12:41
@ti-chi-bot ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 19, 2022
@sdojjy
Copy link
Member

sdojjy commented Jan 19, 2022

/run-unit-test

@sdojjy sdojjy closed this Jan 19, 2022
@sdojjy sdojjy force-pushed the cherry-pick-3845-to-release-5.1 branch from c1661df to 8957631 Compare January 19, 2022 13:47
@ti-chi-bot ti-chi-bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus left review comments

Assignees

@sdojjy sdojjy

Labels
cherry-pick-approved Cherry pick PR approved by release team. component/ddl DDL component. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. status/LGT2 Indicates that a PR has LGTM 2. type/bugfix This PR fixes a bug. type/cherry-pick-for-release-5.1 This PR is cherry-picked to release-5.1 from a source PR.
Projects
None yet
Milestone
v5.1.4
Development

Successfully merging this pull request may close these issues.
5 participants
@ti-chi-bot @sdojjy @codecov-commenter @overvenus @VelocityLight