Refactor mixed_semi_join using cuco::static_set #16230
Merged
+109
−91
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
srinivasyadav18
added
improvement
PointKernel
reviewed
Jul 9, 2024
As rapids-cmake uses the latest cuco which uses key_equal as lhs, thus build and probe table needed to swaped accordingly.
JayjeetAtGithub
previously requested changes
Jul 18, 2024
karthikeyann
reviewed
Aug 6, 2024
| @@ -160,22 +121,16 @@ std::unique_ptr<rmm::device_uvector<size_type>> mixed_join_semi( | |||
| auto const preprocessed_probe = | |||
| experimental::row::equality::preprocessed_table::create(probe, stream); | |||
| auto const row_comparator = | |||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_probe, preprocessed_build}; | |||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_build, preprocessed_probe}; | |||
There was a problem hiding this comment.
is there a reason for this order change build and probe?
There was a problem hiding this comment.
I need to dig into why but without this change the gtests start failing. Maybe @srinivasyadav18 can provide us a quick insight.
There was a problem hiding this comment.
Update: We changed this order and added swap_tables=true at mixed_join_kernels_semi.cu:+72 (tagged you there) to fix the left and right directionality in the equality in single_expression_equality::operator(). Specifically, the single_expression_equality expects indices in {build, probe} order and we have also reset the hasher and equality functors for the cuco::static_set which swaps the internal left and right directions for {build, probe} hence us using swap_table=true.
I know it's quite confusing. 😅
There was a problem hiding this comment.
This is mainly due to a breaking change in NVIDIA/cuCollections#479 where we always put query keys on the left-hand side for comparison.
Please provide benchmark showing what benefit this PR will bring to us. |
ttnghia
reviewed
Aug 15, 2024
ttnghia
reviewed
Aug 15, 2024
mhaseeb123
requested review from
mhaseeb123
and removed request for
mhaseeb123 and
JayjeetAtGithub
PointKernel
approved these changes
Sep 13, 2024
There was a problem hiding this comment.
One minor improvement otherwise looks great to me.
@mhaseeb123 Thank you for stepping in to help resolve the issue and finalize the PR! 🔥 🔥 🔥
| @@ -160,22 +121,16 @@ std::unique_ptr<rmm::device_uvector<size_type>> mixed_join_semi( | |||
| auto const preprocessed_probe = | |||
| experimental::row::equality::preprocessed_table::create(probe, stream); | |||
| auto const row_comparator = | |||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_probe, preprocessed_build}; | |||
| cudf::experimental::row::equality::two_table_comparator{preprocessed_build, preprocessed_probe}; | |||
There was a problem hiding this comment.
This is mainly due to a breaking change in NVIDIA/cuCollections#479 where we always put query keys on the left-hand side for comparison.
Co-authored-by: Yunsong Wang <[email protected]>
mhaseeb123
added
5 - DO NOT MERGE
|
Performance results from Spark (spark-h). Thanks to @zpuller for running this. Queries of interest: q16 and q94 that use |
ttnghia
approved these changes
Sep 17, 2024
|
Shorter for easy reading:
|
|
/merge |
mhaseeb123
added
5 - Ready to Merge
This was referenced Sep 19, 2024
mhaseeb123
added a commit
that referenced
this pull request
Sep 19, 2024
This reverts commit e68f55c.
rapids-bot bot
pushed a commit
that referenced
this pull request
Sep 20, 2024
Reverting #16230 as this PR leads to #16852. Authors: - Muhammad Haseeb (https://github.com/mhaseeb123) Approvers: - Nghia Truong (https://github.com/ttnghia) - Yunsong Wang (https://github.com/PointKernel) - Bradley Dice (https://github.com/bdice) URL: #16855
3 tasks
rapids-bot bot
pushed a commit
that referenced
this pull request
Sep 26, 2024
This PR reapplies changes from #16230 and adds bug fixes and performance improvements for mixed_semi_join. Authors: - Muhammad Haseeb (https://github.com/mhaseeb123) Approvers: - Yunsong Wang (https://github.com/PointKernel) - MithunR (https://github.com/mythrocks) - Nghia Truong (https://github.com/ttnghia) URL: #16859
copy-pr-bot bot
pushed a commit
that referenced
this pull request
Sep 28, 2024
This PR reapplies changes from #16230 and adds bug fixes and performance improvements for mixed_semi_join. Authors: - Muhammad Haseeb (https://github.com/mhaseeb123) Approvers: - Yunsong Wang (https://github.com/PointKernel) - MithunR (https://github.com/mythrocks) - Nghia Truong (https://github.com/ttnghia) URL: #16859
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR refactors
mixed_semi_joinby replacing cuco legacystatic_mapwith lateststatic_set.Contributes to #12261.
Checklist