Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NotImplementedError: Groupby with lists of columns not yet supported. #75

Closed
mathlf2015 opened this issue Aug 30, 2018 · 2 comments · Fixed by #987
Closed

NotImplementedError: Groupby with lists of columns not yet supported. #75

mathlf2015 opened this issue Aug 30, 2018 · 2 comments · Fixed by #987
Labels
new feature/request 💬 Requests and pull requests for new features
Milestone
0.7.0

Comments

@mathlf2015
Copy link

mathlf2015 commented Aug 30, 2018 •
edited
Loading

System information

  • OS Platform and Distribution (e.g., Linux centos):
  • Modin installed from pip:
  • Modin version 0.1.1:
  • Python version 3.6.6:
  • t1 = t1.groupby(["subid","label"])["cnt"].count().reset_index():

0.1.1

Describe the problem

Groupby with lists of columns not yet supported.

Source code / logs

Waiting for redis server at 127.0.0.1:59835 to respond...
Waiting for redis server at 127.0.0.1:16671 to respond...
Starting local scheduler with the following resources: {'CPU': 56, 'GPU': 4}.
Traceback (most recent call last):
File "gen_fea_online.py", line 318, in
df = get_data_log_hive(pre_time=(2018,8,7))
File "gen_fea_online.py", line 290, in get_data_log_hive
df_vod40 = gen_fea_active_log(df_vod40)
File "gen_fea_online.py", line 238, in gen_fea_active_log
t1 = t1.groupby(["subid","label"])["cnt"].count().reset_index()
File "/root/anaconda3/lib/python3.6/site-packages/modin/pandas/dataframe.py", line 823, in groupby
"Groupby with lists of columns not yet supported.")
NotImplementedError: Groupby with lists of columns not yet supported.
@devin-petersohn devin-petersohn added the new feature/request 💬 Requests and pull requests for new features label Aug 30, 2018
@devin-petersohn
Copy link
Collaborator

Hi @mathlf2015, thanks for the request.

We are currently rewriting the majority of the backend for scalability, performance, and maintainability. With this update, groupby will have better performance, and we should be able to extend it to this. #70 contains the rewrite.

@mathlf2015
Copy link
Author

mathlf2015 commented Aug 31, 2018 •
edited
Loading

thank you for your reply.

@jangorecki jangorecki mentioned this issue Sep 30, 2018
Open
@devin-petersohn devin-petersohn mentioned this issue Jun 5, 2019
Closed
22 tasks
@devin-petersohn devin-petersohn added this to the 0.6.4 milestone Dec 19, 2019
devin-petersohn referenced this issue in devin-petersohn/modin Jan 13, 2020
@devin-petersohn
Add support for grouping by multiple columns when doing a reduction
0a7ac42 
* Resolves #75
* Adds support for grouping by multiple columns.
* Does this grouping by broadcasting the columns.
  * A preliminary performance evaluation shows that it is significantly
    faster than before, but still has some room for improvement.
* Minimal code changes to add this new feature.
* We still default to pandas when the user is looping over the dataframe
  * Even though this is common, it is exceptionally hard to optimize,
    and out of scope for this PR.
@devin-petersohn devin-petersohn mentioned this issue Jan 13, 2020
Merged
3 tasks
devin-petersohn added a commit that referenced this issue Jan 13, 2020
@devin-petersohn
Add support for grouping by multiple columns when doing a reduct… (#987)
08f9af5 
* Resolves #75
* Adds support for grouping by multiple columns.
* Does this grouping by broadcasting the columns.
  * A preliminary performance evaluation shows that it is significantly
    faster than before, but still has some room for improvement.
* Minimal code changes to add this new feature.
* We still default to pandas when the user is looping over the dataframe
  * Even though this is common, it is exceptionally hard to optimize,
    and out of scope for this PR.
dchigarev pushed a commit to dchigarev/modin that referenced this issue Aug 25, 2020
@aregm
Merge pull request modin-project#75 from devin-petersohn/groupby/agg
9fb0b8f 
REFACTOR-modin-project#1879: Move logic for `groupby.agg` into query compiler
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Feb 27, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
88b94ac 
…roject#75) [core]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Feb 27, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
4341ab2 
…roject#75) [service]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
mvashishtha added a commit to mvashishtha/modin that referenced this issue Feb 27, 2023
@mvashishtha
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
6f45431 
…roject#75)

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
60d682a 
…roject#75) [core]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
a810f58 
…roject#75) [service]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
5060b54 
…roject#75) [core]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
2f8f805 
…roject#75) [service]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
d6f74de 
…roject#75) [core]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
7bae0b8 
…roject#75) [service]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
08023f6 
…roject#75) [core]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
vnlitvinov pushed a commit to vnlitvinov/modin that referenced this issue Mar 16, 2023
@mvashishtha @vnlitvinov
FIX: Fix API layer implementation of reindex_like [upstream] (modin-p…
cfe246f 
…roject#75) [service]

Note for upstream and rebase: Don't worry about the client query compiler change
here as it's just deleting an unused method. Don't need to upstream or add into
the next modin branch on the next rebase.

Signed-off-by: mvashishtha <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new feature/request 💬 Requests and pull requests for new features
Projects
None yet
Milestone
0.7.0
Development

Successfully merging a pull request may close this issue.

Add support for grouping by multiple columns when doing a reduction devin-petersohn/modin
2 participants
@devin-petersohn @mathlf2015