-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs/design: add proposal for skyline pruning #9184
Changes from 1 commit
eeb75a1
6045645
5969b01
a9b8fad
e7d3055
3ce656c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
# Proposal: Support Skyline Pruning | ||
|
||
- Author(s): [Haibin Xie](https://github.com/lamxTyler) | ||
- Last updated: 2019-01-25 | ||
- Discussion at: | ||
|
||
## Abstract | ||
|
||
This proposal introduces some heuristics and a general framework for pruning the access paths. With the help of it, the optimizer can avoid some wrong choices on access paths. | ||
|
||
## Background | ||
|
||
Currently, the choice of access path strongly depends on the statistics. We may choose the wrong index due to outdated statistics. However, many of the wrong choices can be eliminated by simple heuristics, for example, if the primary key or unique indices can be fully matched, we can choose it without the referring to the statistics. | ||
|
||
## Proposal | ||
|
||
The most important factors to choose the access paths are the number of rows that need to be scanned, whether or not it matches the physical property and does it require a double scan. Among these three factors, only the scan row count depends on statistics. So how can we compare the scan row count without statistics? Let's take a look at an example: | ||
|
||
```sql | ||
create table t(a int, b int, c int, index idx1(b, a), index idx2(a)); | ||
select * from t where a = 1 and b = 1; | ||
``` | ||
|
||
From the query and schema, we can know that the access condition of `idx1` could strictly covers `idx2`, therefore the number of rows scanned by `idx1` will be no more than `idx2`, so `idx1` will be better than `idx2` in this case. | ||
alivxxx marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
So how can we combine these factors to prune the access paths? Consider two access paths `x` and `y`, if `x` is not worse than `y` at all factors, and there exists one factor that `x` is better than `y`, then we can prune `y` before referring to the statistics, because `x` will works better than `y` at all circumstances. This is also called skyline pruning. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you add all the rules that you decided to implement in the proposed skyline pruning process? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you mean a detailed explanation of the three factors at https://github.com/pingcap/tidb/pull/9184/files#diff-136059bc4631ad80be06dc8299b85b6dR17? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, actually I mean this:
For now, there is only one rule:
Is there any other rule? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. All the three factors will be rules, but the rule of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For the first factor There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, currently there is only one rule for the first factor.
alivxxx marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Rationale | ||
|
||
The skyline pruning is also implemented on other databases, including MySQL and OceanBase. Without it, we may suffer from choosing the wrong access path in some simple cases. | ||
alivxxx marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
## Compatibility | ||
|
||
It does not affect the compatibility. | ||
|
||
## Implementation | ||
|
||
Since we need to decide whether the access path matches the physical property, we need to do the skyline pruning when finding the best task for the data source. And usually there won't be too many indices, a naive nested-loops algorithm will suffice. The comparison of any two access paths has been explained in the previous `Proposal` section. | ||
|
||
## Open issues (if applicable) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do these two factors have different priorities?
single read/ double read
required properties
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, they are equal.