Feat sort merge join #668
Feat sort merge join #668
Conversation
Codecov ReportPatch coverage:
❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more. Additional details and impacted files@@ Coverage Diff @@
## master #668 +/- ##
==========================================
+ Coverage 36.77% 37.29% +0.52%
==========================================
Files 229 230 +1
Lines 24016 24429 +413
==========================================
+ Hits 8832 9112 +280
- Misses 14177 14273 +96
- Partials 1007 1044 +37
☔ View full report in Codecov by Sentry. |
|
|
||
| var outerValue, innerValue proto.Value | ||
|
|
||
| for { |
There was a problem hiding this comment.
- should 'yield' current compare checkpoint rows, consider: [1,2,3] inner join [2,2,2,3,3,3].
- should handle all join types, not only inner join.
|
Please add unit tests. |
|
|
||
| func (s *SortMergeJoin) Next() (proto.Row, error) { | ||
| // all data is order | ||
|
|
There was a problem hiding this comment.
please remove the unnecessary blank line at the function beginning.
pkg/dataset/sort_merge_join.go
Outdated
| func (s *SortMergeJoin) resGenerate(leftRow proto.Row, rightRow proto.Row) proto.Row { | ||
| res := make([]proto.Value, leftRow.Length()+rightRow.Length()) | ||
| leftValue := make([]proto.Value, leftRow.Length()) | ||
| rightValue := make([]proto.Value, rightRow.Length()) |
There was a problem hiding this comment.
if rightRow is nil(refer to line 100), would this cause NPE?
pkg/dataset/sort_merge_join.go
Outdated
| } | ||
|
|
||
| func (s *SortMergeJoin) Next() (proto.Row, error) { | ||
| // all data is order |
There was a problem hiding this comment.
Maybe it's better to move the comment into the SortMergeJoin struct like this:
//SortMergeJoin ensures that the data sets participating in the join are in order before using sort-merge join.
type SortMergeJoin struct {
fields []proto.Field
joinColumn *JoinColumn
joinType ast.JoinType
outer proto.Dataset
inner proto.Dataset
}
|
SonarCloud Quality Gate failed.
|
pkg/dataset/sort_merge_join.go
Outdated
| package dataset | ||
|
|
||
| import ( | ||
| "go.uber.org/atomic" |
There was a problem hiding this comment.
@wang1309 at root package, use this tool:
go get -u github.com/dubbogo/tools/cmd/imports-formatter
imports-formatter
|
SonarCloud Quality Gate failed.
|
|
Kudos, SonarCloud Quality Gate passed!
|
* add sort merge join file * feat:add sort merge join * feat: sort merge join feature add * temp commit * add unit test and optimize code * delete unuse file * fixed: fix review dog fault * add comment * format import --------- Co-authored-by: wangrui130 <[email protected]>
What this PR does:
sort merge join dataset
Which issue(s) this PR fixes:
Fixes #642
Special notes for your reviewer:
Does this PR introduce a user-facing change?: