Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddl: invalid multiple MAXVALUE partitions (#36329) #36345

Merged
merged 4 commits into from
Jul 21, 2022

Conversation

u5surf
Copy link
Contributor

@u5surf u5surf commented Jul 19, 2022

Signed-off-by: u5surf [email protected]

What problem does this PR solve?

Issue Number: close #36329

Problem Summary:

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release-note

None

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Jul 19, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • Defined2014
  • mjonss

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 19, 2022
@u5surf
Copy link
Contributor Author

u5surf commented Jul 20, 2022

/cc @mjonss

@ti-chi-bot ti-chi-bot requested a review from mjonss July 20, 2022 01:17
Copy link
Contributor

@mjonss mjonss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for single column RANGE COLUMNS (which is what we currently support), but it will not work for multi-column RANGE COLUMNS, like this:

create table t (a int, b int, c int) partition by range columns (a,b,c)
(partition p0 values less than (10, MAXVALUE, 0),
 partition p1 values less than (10, MAXVALUE, 1000))

(which also fails in MySQL, but I consider it to be a bug.)

ddl/ddl_api.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Jul 20, 2022
@mjonss
Copy link
Contributor

mjonss commented Jul 20, 2022

/run-all-tests

"partition by range columns (d) (" +
"partition p0 values less than ('2022-01-01')," +
"partition p1 values less than (MAXVALUE), " +
"partition p2 values less than (MAXVALUE));",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please format this line, I think you can just do make dev and it will reformat it for you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mjonss reformatted with make dev

@sre-bot
Copy link
Contributor

sre-bot commented Jul 20, 2022

ddl/ddl_api.go Outdated
@@ -2762,8 +2762,10 @@ func checkTwoRangeColumns(ctx sessionctx.Context, curr, prev *model.PartitionDef
for i := 0; i < len(pi.Columns); i++ {
// Special handling for MAXVALUE.
if strings.EqualFold(curr.LessThan[i], partitionMaxValue) {
// If current is maxvalue, it certainly >= previous.
return true, nil
if !strings.EqualFold(prev.LessThan[i], partitionMaxValue) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think combine 2764 and 2765 in one line is better.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Defined2014 Exactly, fixed.

ddl/ddl_api.go Outdated
// If current is maxvalue, it certainly >= previous.
return true, nil

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Defined2014 my bad.. Fixed.

@ti-srebot
Copy link
Contributor

cherry pick to release-5.3 in PR #36444

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jul 21, 2022
@ti-srebot
Copy link
Contributor

cherry pick to release-5.4 in PR #36445

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jul 21, 2022
@ti-srebot
Copy link
Contributor

cherry pick to release-6.0 in PR #36446

ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Jul 21, 2022
@ti-srebot
Copy link
Contributor

cherry pick to release-6.1 in PR #36447

@ti-srebot
Copy link
Contributor

cherry pick to release-6.2 failed

@sre-bot
Copy link
Contributor

sre-bot commented Jul 22, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 failed 1, success 0, total 1 17 min New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 failed 6, success 0, total 6 12 min New failing
idc-jenkins-ci-tidb/common-test 🟥 all 0 tests passed 10 min New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 5 min 51 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 2 min 52 sec New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 10, success 1, total 11 38 min Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 20 min Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 0 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 19 sec Existing passed

morgo added a commit to morgo/tidb that referenced this pull request Jul 22, 2022
…ip-init

* upstream/master: (125 commits)
  infoschema: fix PromQL for `tidb_distsql_copr_cache` (pingcap#36450)
  test: stabilize TestTopSQLCPUProfile (pingcap#36468)
  parser: add support of 'ADMIN SHOW DDL JOB QUERIES LIMIT m OFFSET n' transferring to AST (pingcap#36285)
  *: enable flaky test for all test (pingcap#36385)
  expression: fix return type of agg func `bit_or` when handling varbinary column (pingcap#36415)
  executor: fix aggregating enum zero value gets different results from mysql  (pingcap#36208)
  server: skip check tiflash version (pingcap#36451)
  *: Minor update to SECURITY.md to improved clarity (pingcap#36346)
  table partition: add telemetry for partition table (pingcap#36204)
  ddl: invalid multiple MAXVALUE partitions (pingcap#36329) (pingcap#36345)
  planner: Fixed `Merge` hint in nested CTE (pingcap#36432)
  metric: impove concurrency ddl metrics (pingcap#36405)
  planner: add more test cases for leading outer join (pingcap#36409)
  ddl: only set concurrent variable if no error (pingcap#36437)
  ddl: fix update panic in the middle of multi-schema change (pingcap#36421)
  session: Mising OptimizeWithPlanAndThenWarmUp in prepare-execute path (pingcap#36347)
  executor,metrics: add a metric for observing execution phases (pingcap#35906)
  br: unified docker image align with tidb (pingcap#36016)
  ddl: skip to close nil sessPool  (pingcap#36425)
  log-backup: remove the timezone from log-date (pingcap#36369)
  ...
morgo added a commit to morgo/tidb that referenced this pull request Jul 22, 2022
* upstream/master: (280 commits)
  infoschema: fix PromQL for `tidb_distsql_copr_cache` (pingcap#36450)
  test: stabilize TestTopSQLCPUProfile (pingcap#36468)
  parser: add support of 'ADMIN SHOW DDL JOB QUERIES LIMIT m OFFSET n' transferring to AST (pingcap#36285)
  *: enable flaky test for all test (pingcap#36385)
  expression: fix return type of agg func `bit_or` when handling varbinary column (pingcap#36415)
  executor: fix aggregating enum zero value gets different results from mysql  (pingcap#36208)
  server: skip check tiflash version (pingcap#36451)
  *: Minor update to SECURITY.md to improved clarity (pingcap#36346)
  table partition: add telemetry for partition table (pingcap#36204)
  ddl: invalid multiple MAXVALUE partitions (pingcap#36329) (pingcap#36345)
  planner: Fixed `Merge` hint in nested CTE (pingcap#36432)
  metric: impove concurrency ddl metrics (pingcap#36405)
  planner: add more test cases for leading outer join (pingcap#36409)
  ddl: only set concurrent variable if no error (pingcap#36437)
  ddl: fix update panic in the middle of multi-schema change (pingcap#36421)
  session: Mising OptimizeWithPlanAndThenWarmUp in prepare-execute path (pingcap#36347)
  executor,metrics: add a metric for observing execution phases (pingcap#35906)
  br: unified docker image align with tidb (pingcap#36016)
  ddl: skip to close nil sessPool  (pingcap#36425)
  log-backup: remove the timezone from log-date (pingcap#36369)
  ...
joycse06 added a commit to joycse06/tidb that referenced this pull request Jul 23, 2022
…rimary-key

* upstream/master: (104 commits)
  br: fix compatibility issue with concurrent ddl (pingcap#36474)
  infoschema: fix PromQL for `tidb_distsql_copr_cache` (pingcap#36450)
  test: stabilize TestTopSQLCPUProfile (pingcap#36468)
  parser: add support of 'ADMIN SHOW DDL JOB QUERIES LIMIT m OFFSET n' transferring to AST (pingcap#36285)
  *: enable flaky test for all test (pingcap#36385)
  expression: fix return type of agg func `bit_or` when handling varbinary column (pingcap#36415)
  executor: fix aggregating enum zero value gets different results from mysql  (pingcap#36208)
  server: skip check tiflash version (pingcap#36451)
  *: Minor update to SECURITY.md to improved clarity (pingcap#36346)
  table partition: add telemetry for partition table (pingcap#36204)
  ddl: invalid multiple MAXVALUE partitions (pingcap#36329) (pingcap#36345)
  planner: Fixed `Merge` hint in nested CTE (pingcap#36432)
  metric: impove concurrency ddl metrics (pingcap#36405)
  planner: add more test cases for leading outer join (pingcap#36409)
  ddl: only set concurrent variable if no error (pingcap#36437)
  ddl: fix update panic in the middle of multi-schema change (pingcap#36421)
  session: Mising OptimizeWithPlanAndThenWarmUp in prepare-execute path (pingcap#36347)
  executor,metrics: add a metric for observing execution phases (pingcap#35906)
  br: unified docker image align with tidb (pingcap#36016)
  ddl: skip to close nil sessPool  (pingcap#36425)
  ...
@sre-bot
Copy link
Contributor

sre-bot commented Aug 3, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 3 min 52 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 4 min 2 sec New failing
idc-jenkins-ci-tidb/common-test 🟥 all 11 tests passed 4 min 12 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 failed 13, success 0, total 13 3 min 44 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 all 29 tests passed 26 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 all 11 tests passed 8 min 12 sec Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 failed 28, success 0, total 28 2 min 46 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 48 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 10 min Existing passed

@sre-bot
Copy link
Contributor

sre-bot commented Aug 3, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 5 min 27 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 3 min 26 sec New failing
idc-jenkins-ci-tidb/common-test 🟥 failed 1, success 10, total 11 3 min 50 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 all 13 tests passed 4 min 55 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 all 29 tests passed 26 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 1, success 10, total 11 5 min 42 sec Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 5 min 29 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 21 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 9 min 28 sec Existing passed

@sre-bot
Copy link
Contributor

sre-bot commented Aug 3, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 5 min 39 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 9 min 9 sec New failing
idc-jenkins-ci-tidb/common-test 🟥 failed 1, success 10, total 11 6 min 7 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 all 13 tests passed 7 min 6 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 all 31 tests passed 29 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 all 11 tests passed 8 min 53 sec Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 8 min 37 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 4 min 7 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 10 min Existing passed

@sre-bot
Copy link
Contributor

sre-bot commented Aug 3, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 3 min 7 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 14 min New failing
idc-jenkins-ci-tidb/common-test 🟥 all 11 tests passed 9 min 18 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 all 13 tests passed 7 min 29 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 all 35 tests passed 33 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 1, success 10, total 11 18 min Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 12 min Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 7 min 26 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 19 sec Existing passed

@sre-bot
Copy link
Contributor

sre-bot commented Aug 4, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 2 min 48 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 6 min 20 sec New failing
idc-jenkins-ci-tidb/common-test 🟥 all 11 tests passed 5 min 30 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 all 13 tests passed 5 min 3 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 all 31 tests passed 28 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 failed 1, success 10, total 11 9 min 21 sec Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 6 min 10 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 3 min 9 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 8 min 54 sec Existing passed

@sre-bot
Copy link
Contributor

sre-bot commented Aug 4, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [5] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/integration-compatibility-test 🟥 all 1 tests passed 2 min 41 sec New failing
idc-jenkins-ci-tidb/integration-ddl-test 🟥 all 6 tests passed 3 min 45 sec New failing
idc-jenkins-ci-tidb/common-test 🟥 all 11 tests passed 3 min 35 sec New failing
idc-jenkins-ci-tidb/sqllogic-test-1 🟥 all 13 tests passed 4 min 38 sec New failing
idc-jenkins-ci/integration-cdc-test 🟥 failed 1, success 28, total 29 17 min New failing
idc-jenkins-ci-tidb/integration-common-test 🔴 all 11 tests passed 7 min 0 sec Existing failure
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 4 min 45 sec Existing passed
idc-jenkins-ci-tidb/mybatis-test 🟢 all 1 tests passed 2 min 54 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 8 min 46 sec Existing passed

@u5surf
Copy link
Contributor Author

u5surf commented Aug 4, 2022

Hi, @Defined2014. The CI failed alert won't stop ringing now, what should I do?

@Defined2014
Copy link
Contributor

@u5surf I think you can ignore this. We cherry-pick this commit to other release branch with broken CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-cherry-pick-release-5.0 needs-cherry-pick-release-5.1 needs-cherry-pick-release-5.2 needs-cherry-pick-release-5.3 Type: Need cherry pick to release-5.3 needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.1 Should cherry pick this PR to release-6.1 branch. release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PARTITION BY RANGE COLUMNS allows multiple MAXVALUE partitions
6 participants