executor, expression: add a tryToMatchOuters for joiner #11922
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
XuHuaiyu
added
type/enhancement
Codecov Report
@@ Coverage Diff @@
## master #11922 +/- ##
===========================================
Coverage 81.5083% 81.5083%
===========================================
Files 449 449
Lines 97390 97390
===========================================
Hits 79381 79381
Misses 12385 12385
Partials 5624 5624 |
Closed
|
/run-all-tests |
SunRunAway
reviewed
Sep 4, 2019
SunRunAway
reviewed
Sep 5, 2019
zz-jason
reviewed
Sep 5, 2019
| } | ||
| outerRowStatus = append(outerRowStatus, outerRowMatched) | ||
| if matched { | ||
| chk.AppendPartialRow(0, outer) |
There was a problem hiding this comment.
Should we only decrease numToAppend when this operation is performed?
There was a problem hiding this comment.
No~ outers.Next is needed.
executor/joiner.go
Outdated
| } | ||
| return outerRowStatus, nil | ||
| } | ||
| for outer, i := outers.Current(), 0; outer != outers.End() && numToAppend > 0; outer, numToAppend, i = outers.Next(), numToAppend-1, i+1 { |
executor/joiner.go
Outdated
| return outerRowStatus, nil | ||
| } | ||
|
|
||
| for i := 0; outer != outers.End() && numToAppend > 0; outer, numToAppend, i = outers.Next(), numToAppend-1, i+1 { |
|
PTAL @zz-jason |
qw4990
added
status/can-merge
zz-jason
reviewed
Sep 10, 2019
| task.outerRowStatus[rowIdx] = outerRowHasNull | ||
| for _, status := range iw.outerRowStatus { | ||
| outerRowIdx := matchedOuterRowIdx[cursor] | ||
| if status == outerRowMatched || task.outerRowStatus[outerRowIdx] == outerRowUnmatched { |
There was a problem hiding this comment.
can it be replaced with the following condition?
if task.outerRowStatus[outerRowIdx] == outerRowUnMatched {
task.outerRowStatus[outerRowIdx] = status
}There was a problem hiding this comment.
No
If so, when task.outerRowStatus[outerRowIdx] == outerRowHasNull && status == outerRowMatched is true, it will cause error.
| // returns two bool slices, `selected` indicates whether a row passed the | ||
| // filters, `isNull` indicates whether the result of the filter is null. | ||
| // Filters is executed vectorized. | ||
| func VectorizedFilterConsiderNull(ctx sessionctx.Context, filters []Expression, iterator *chunk.Iterator4Chunk, selected []bool, isNil []bool) ([]bool, []bool, error) { |
There was a problem hiding this comment.
- s/isNil/isNull/
- Can we refactor
VectorizedFilter()to callVectorizedFilterConsiderNull()?
| // tryToMatchOuters, the result is built by multiple outer rows and one inner | ||
| // row. The returned outerRowStatusFlag slice value indicates the status of | ||
| // each outer row (matched/unmatched/hasNull). | ||
| func (j *baseJoiner) filterAndCheckOuterRowStatus(input, output *chunk.Chunk, innerColsLen int, outerRowStatus []outerRowStatusFlag) (_ []outerRowStatusFlag, _ error) { |
There was a problem hiding this comment.
can we refactor it to call j.Filter() then update outerRowStatus?
There was a problem hiding this comment.
The code already did this?
| } | ||
|
|
||
| outer, numToAppend, outerBatchSize := outers.Current(), chk.RequiredRows()-chk.NumRows(), 0 | ||
| for ; outer != outers.End() && numToAppend > 0; outer, numToAppend, outerBatchSize = outers.Next(), numToAppend-1, outerBatchSize+1 { |
There was a problem hiding this comment.
Don't need to calculate outerBatchSize in every loop. We can calculate it when the loop is finished:
outer, numToAppend := outers.Current(), chk.RequiredRows()-chk.NumRows()
for ; outer != outers.End() && numToAppend > 0; outer, numToAppend = outers.Next(), numToAppend-1 {
j.makeJoinRowToChunk(chkForJoin, outer, inner)
}
outerBatchSize := chk.RequiredRows() - chk.NumRows() - numToAppend
// do somethingBTW, the for loop looks a little complicated, we can simplify it to:
for outer != outers.End() && numToAppend > 0 {
j.makeJoinRowToChunk(chkForJoin, outer, inner)
outer = outers.Next()
numToAppend = numToAppend - 1
}
|
/run-all-tests |
|
@zz-jason This commit is merged... |
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Add a tryToMatchOuters interface in joiner which joins a batch out outer rows and one inner row every time.
What is changed and how it works?
Check List
Tests
Exist tests.
More test case needs to be added.
Code changes
Side effects
N/A
Related changes
N/A
Release note
N/A