-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add drop_columns to dataframe api #11010
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @Omega359
I left a little suggestion about the container's type.
.schema() | ||
.qualified_field_with_unqualified_name(name) | ||
}) | ||
.collect::<Result<Vec<_>>>()?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
collect into a hash table might be good for wide table scenarios
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, I'll take a look. The logic for this function was based on the select_columns fn which did things that way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @Omega359
Appreciate if you add more tests, like drop with empty array
Drop where columns are with double quotes, drop non existent cols, drop duplicated
* Add drop_columns to dataframe api apache#11007 * Prettier cleanup * Added additional drop_columns tests and fixed issue with nonexistent columns.
* Add drop_columns to dataframe api apache#11007 * Prettier cleanup * Added additional drop_columns tests and fixed issue with nonexistent columns.
* Add drop_columns to dataframe api apache#11007 * Prettier cleanup * Added additional drop_columns tests and fixed issue with nonexistent columns.
Which issue does this PR close?
Closes #11007
Rationale for this change
Add drop_columns to the dataframe api.
What changes are included in this PR?
code, tests, documentation
Are these changes tested?
Yes via tests in dataframe/mod.rs
Are there any user-facing changes?
dataframe API was amended.