Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement cat.codes using map rather than full-axis approach #6224

Closed
dchigarev opened this issue May 31, 2023 · 0 comments · Fixed by #6230
Closed

Implement cat.codes using map rather than full-axis approach #6224

dchigarev opened this issue May 31, 2023 · 0 comments · Fixed by #6230
Assignees
@dchigarev
Labels
new feature/request 💬 Requests and pull requests for new features P1 Important tasks that we should complete soon

Comments

@dchigarev
Copy link
Collaborator

After the #6222 is merged, every partition now knows the global categorical codes, meaning that there's no need to use the full-axis fold anymore as we can now simply map over partitions and still receive valid codes

modin/modin/core/storage_formats/pandas/query_compiler.py

Lines 4043 to 4045 in 7029a70

res = self._modin_frame.fold(
axis=0, func=func, new_columns=[MODIN_UNNAMED_SERIES_LABEL]
)

@dchigarev dchigarev added new feature/request 💬 Requests and pull requests for new features Triage 🩹 Issues that need triage labels May 31, 2023
@dchigarev dchigarev self-assigned this May 31, 2023
@dchigarev dchigarev added P1 Important tasks that we should complete soon and removed Triage 🩹 Issues that need triage labels May 31, 2023
dchigarev added a commit to dchigarev/modin that referenced this issue Jun 1, 2023
@dchigarev
PERF-modin-project#6224: Use 'Map' operator to retrieve categorical c…
85399d9 
…odes

Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev dchigarev mentioned this issue Jun 1, 2023
Merged
7 tasks
YarShev pushed a commit that referenced this issue Jun 1, 2023
@dchigarev
PERF-#6224: Use 'Map' operator to retrieve categorical codes (#6230)
189ae67 
Signed-off-by: Dmitry Chigarev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dchigarev dchigarev

Labels
new feature/request 💬 Requests and pull requests for new features P1 Important tasks that we should complete soon
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

PERF-#6224: Use 'Map' operator to retrieve categorical codes dchigarev/modin
1 participant
@dchigarev