Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] collate_fn in iter_torch_batches could be a bottleneck #33508

Closed
llan-ml opened this issue Mar 21, 2023 · 4 comments
Closed

[Data] collate_fn in iter_torch_batches could be a bottleneck #33508

llan-ml opened this issue Mar 21, 2023 · 4 comments
Labels
data Ray Data-related issues enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks

Comments

@llan-ml
Copy link
Contributor

llan-ml commented Mar 21, 2023

Description

The current implementation does custom batching in a sequential manner. In some cases that the user-defined collate_fn consumes much time, the overall dataset pipeline is still slow. In contrast, the collate_fn in pytorch DataLoader executes in parallel.

Use case

No response
@llan-ml llan-ml added enhancement Request for new feature and/or capability triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Mar 21, 2023
@amogkam
Copy link
Contributor

amogkam commented Mar 21, 2023

this will be fixed by #33510!

the _collate_fn will run in a threadpool

@Yard1 Yard1 mentioned this issue Mar 22, 2023
Closed
10 tasks
@scottjlee scottjlee added P1 Issue that should be fixed within a few weeks data Ray Data-related issues and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Mar 24, 2023
@raulchen
Copy link
Contributor

@amogkam #33510 was closed. Anything else we can do for this issue?

@genesis-jamin
Copy link

genesis-jamin commented Nov 20, 2023 •
edited
Loading

Was this feature ever implemented? #33510 seems to have been closed without merging, and from viewing the flame graph I'm seeing significant time spent in my custom collate function.

@justinvyu
Copy link
Contributor

@genesis-jamin The linked PR was a prototype that got split into multiple PRs that have already been merged.

  1. [Data] [1/n] Async iter_batches: Add Threadpool util #33575
  2. [Data] [2/3] Async iter_batches: Add interfaces and helper functions #33605
  3. [data] [3/3] [no_early_kickoff] Async iter batches e2e #33620

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
data Ray Data-related issues enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Data] Async iter_batches amogkam/ray
7 participants
@raulchen @justinvyu @scottjlee @amogkam @llan-ml @genesis-jamin @anyscalesam