Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[data] [streaming] Enable streaming executor by default #32132

Closed
ericl opened this issue Jan 31, 2023 · 1 comment
Closed

[data] [streaming] Enable streaming executor by default #32132

ericl opened this issue Jan 31, 2023 · 1 comment
Assignees
@jianoaix
Labels
data Ray Data-related issues enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks Ray 2.4
Milestone
Dataset Streaming...

Comments

@ericl
Copy link
Contributor

ericl commented Jan 31, 2023

Flip the feature flag for streaming executor by default, and fix any issues that occur.
@ericl ericl added enhancement Request for new feature and/or capability P2 Important issue, but not time-critical data Ray Data-related issues Ray 2.4 labels Jan 31, 2023
@ericl ericl added this to the Dataset Streaming Execution milestone Jan 31, 2023
@ericl ericl added P1 Issue that should be fixed within a few weeks and removed P2 Important issue, but not time-critical labels Feb 6, 2023
@jianoaix
Copy link
Contributor

jianoaix commented Feb 16, 2023 •
edited
Loading

Item list to resolve (updating in progress):

@jianoaix jianoaix mentioned this issue Feb 17, 2023
Merged
7 tasks
ericl pushed a commit that referenced this issue Feb 22, 2023
@jianoaix
Streaming executor fixes (#32658)
875a555 
- current resource can have none object store memory (we don't account input buffer)
- We need at least 2 CPUs to run a actorpool streaming
- We no longer use LazyBlockList and cache partial results in streaming executor

#32132
This was referenced Feb 22, 2023
Merged
Merged
Merged
@jianoaix jianoaix mentioned this issue Mar 2, 2023
Merged
7 tasks
@ericl ericl closed this as completed Mar 20, 2023
edoakes pushed a commit to edoakes/ray that referenced this issue Mar 22, 2023
@jianoaix @edoakes
Streaming executor fixes (ray-project#32658)
270b43f 
- current resource can have none object store memory (we don't account input buffer)
- We need at least 2 CPUs to run a actorpool streaming
- We no longer use LazyBlockList and cache partial results in streaming executor

ray-project#32132
Signed-off-by: Edward Oakes <[email protected]>
peytondmurray pushed a commit to peytondmurray/ray that referenced this issue Mar 22, 2023
@jianoaix @peytondmurray
Streaming executor fixes (ray-project#32658)
21e36a0 
- current resource can have none object store memory (we don't account input buffer)
- We need at least 2 CPUs to run a actorpool streaming
- We no longer use LazyBlockList and cache partial results in streaming executor

ray-project#32132
elliottower pushed a commit to elliottower/ray that referenced this issue Apr 22, 2023
@jianoaix @elliottower
Streaming executor fixes (ray-project#32658)
4ea0e27 
- current resource can have none object store memory (we don't account input buffer)
- We need at least 2 CPUs to run a actorpool streaming
- We no longer use LazyBlockList and cache partial results in streaming executor

ray-project#32132
Signed-off-by: elliottower <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jianoaix jianoaix

Labels
data Ray Data-related issues enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks Ray 2.4
Projects
None yet
Milestone
Dataset Streaming Execution
Development

No branches or pull requests
2 participants
@ericl @jianoaix