Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Partition table improvement #18016

Closed
4 tasks
scsldb opened this issue Jun 15, 2020 · 1 comment
Closed
4 tasks

Partition table improvement #18016

scsldb opened this issue Jun 15, 2020 · 1 comment
Assignees
@lysu
Labels
feature/accepted This feature request is accepted by product managers priority/P0 The issue has P0 priority. type/feature-request Categorizes issue or PR as related to a new feature.

Comments

@scsldb
Copy link

scsldb commented Jun 15, 2020 •
edited
Loading

Description

We want partition table have better performance than before. The current plan is pruning partition table in build physical plan phase rather than in optimize logical plan phase, and replace statistic in each partition into a whole table statistic information. This two step will reduce the CPU use in partition table. Partition pruning algorithm will also be improved, support pruning more condition, including in and or for hash partition.

This project containing following tasks:

  • P0: #18546 Implement PartitionTableReader to support pruning during execute stage
  • P0: #18548 Improve partition pruning algorithm to support more operator
  • P0: #18551 Implement table level statistic and index selection to support move pruning to execute stage and reduce performance degrade when cluster has many partitions
  • Testing: compare the new partition table with old one using TPC-C and sysbench workload to verify performance improving.

Progress Tracking: https://github.com/pingcap/tidb/projects/52#card-41891412

Category

Performance & Stability

Value

Improve system performance & support more operation on the partition table

Time

GanttProgress: 96%
GanttStart: 2020-07-15
GanttDue: 2020-10-20
@scsldb scsldb added type/feature-request Categorizes issue or PR as related to a new feature. priority/P0 The issue has P0 priority. labels Jun 15, 2020
@scsldb scsldb added this to the v5.0-alpha milestone Jun 15, 2020
@scsldb scsldb changed the title Optimize the hot issues of Partition table Improve partition pruning and data access performance on partition table Jun 30, 2020
@jackysp jackysp assigned imtbkcat and unassigned jackysp and zhangjinpeng87 Jun 30, 2020
@zz-jason zz-jason mentioned this issue Jul 10, 2020
Closed
@zz-jason zz-jason mentioned this issue Jul 15, 2020
Closed
6 tasks
@scsldb scsldb added the feature/accepted This feature request is accepted by product managers label Jul 16, 2020
@cfzjywxk cfzjywxk changed the title Improve partition pruning and data access performance on partition table Partition table improvement Sep 18, 2020
@cfzjywxk cfzjywxk assigned lysu and unassigned imtbkcat Sep 18, 2020
@jebter jebter modified the milestones: v5.0.0-alpha, v5.0.0-rc Jan 7, 2021
@jebter jebter modified the milestones: v5.0.0-rc, v5.0.0 Jan 18, 2021
@zz-jason zz-jason removed this from the v5.0.0 ga milestone Apr 20, 2021
@qw4990
Copy link
Contributor

qw4990 commented Apr 22, 2021

Close it since it is duplicated with #24150.

@qw4990 qw4990 closed this as completed Apr 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lysu lysu

Labels
feature/accepted This feature request is accepted by product managers priority/P0 The issue has P0 priority. type/feature-request Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@lysu @jackysp @jebter @zz-jason @scsldb @qw4990 @imtbkcat @zhangjinpeng87