Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Add nested struct support for cudf::contains #8965

Closed
revans2 opened this issue Aug 5, 2021 · 4 comments · Fixed by #10656
Closed

[FEA] Add nested struct support for cudf::contains #8965

revans2 opened this issue Aug 5, 2021 · 4 comments · Fixed by #10656
Assignees
@ttnghia
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Milestone
List and Struct d...

Comments

@revans2
Copy link
Contributor

revans2 commented Aug 5, 2021 •
edited
Loading

Is your feature request related to a problem? Please describe.
For Spark we are pushing to get more support for structs in a number of operators. We already have some support for sorting structs, so we should be able to come up with a way to check for existence of a struct value from one column to another. NOTE this does not include lists as children of the structs just structs that contains basic types including strings and other structs.

his should follow the same pattern we have supported for sorting where null child columns are considered equal to other null child columns. Like described in #8964

Describe the solution you'd like
I would like to see cudf::contains updated so it can support this.

Describe alternatives you've considered
I don't think there is an alternative that we can do on our own.
@revans2 revans2 added feature request New feature or request Needs Triage Need team to review and classify Spark Functionality that helps Spark RAPIDS labels Aug 5, 2021
@revans2 revans2 mentioned this issue Aug 5, 2021
Open
@beckernick beckernick added libcudf Affects libcudf (C++/CUDA) code. and removed Needs Triage Need team to review and classify labels Aug 25, 2021
@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@ttnghia ttnghia self-assigned this Dec 11, 2021
@ttnghia ttnghia mentioned this issue Dec 17, 2021
Merged
rapids-bot bot pushed a commit that referenced this issue Jan 15, 2022
@ttnghia
Support structs for cudf::contains with column/scalar input (#9929)
7ff5f12 
This PR adds support for `cudf::contains` so we can check whether a structs column contains a scalar struct element.

Partially addresses #8965. This does not support checking if structs given in a structs column exist in another structs column. Such cases will be supported when the new data structure mentioned in #9413 is merged into cudf.

Authors:
  - Nghia Truong (https://github.com/ttnghia)

Approvers:
  - Mike Wilson (https://github.com/hyperbolic2346)
  - MithunR (https://github.com/mythrocks)

URL: #9929
@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@revans2
Copy link
Contributor Author

revans2 commented Mar 11, 2022

This is still wanted

@ttnghia
Copy link
Contributor

ttnghia commented Mar 14, 2022 •
edited
Loading

Please note that this has already been partially supported in #9929
Full support for this requires complete hashing work, which is undergone some refactoring in the meantime.

@ttnghia ttnghia mentioned this issue Apr 14, 2022
Merged
rapids-bot bot pushed a commit that referenced this issue Aug 17, 2022
@ttnghia
Fully support nested types in cudf::contains (#10656)
127d574 
This extends the `cudf::contains` API to support nested types (lists + structs) with arbitrarily nested levels. As such, `cudf::contains` will work with literally any type of input data.

In addition, this fixes null handling of `cudf::contains` with structs column + struct scalar input when the structs column contains null rows at the top level while the scalar key is valid but all nulls at children levels.

Closes: #8965
Depends on:
 * #10730
 * #10883
 * #10802
 * #10997
 * NVIDIA/cuCollections#172
 * NVIDIA/cuCollections#173
 * #11037
 * #11356

Authors:
  - Nghia Truong (https://github.com/ttnghia)
  - Devavret Makkar (https://github.com/devavret)
  - Bradley Dice (https://github.com/bdice)
  - Karthikeyan (https://github.com/karthikeyann)

Approvers:
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Bradley Dice (https://github.com/bdice)
  - Yunsong Wang (https://github.com/PointKernel)

URL: #10656
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ttnghia ttnghia

Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Milestone
List and Struct data types and operat...
Development

Successfully merging a pull request may close this issue.

Fully support nested types in cudf::contains ttnghia/cudf
4 participants
@harrism @revans2 @ttnghia @beckernick