Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Remove Ray Data read parallelism from all libraries and tests #43692

Merged
merged 2 commits into from
Mar 5, 2024
Merged

[Data] Remove Ray Data read parallelism from all libraries and tests #43692

merged 2 commits into from
Mar 5, 2024

Conversation

c21
Copy link
Contributor

@c21 c21 commented Mar 4, 2024

Why are these changes needed?

This PR is to remove Ray Data read parallelism, replace with override_num_blocks from all libraries and unit tests. The PR change on Ray Data is #43113 .

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@c21 c21 added the release-blocker P0 Issue that blocks the release label Mar 4, 2024
@c21 c21 merged commit 9a36755 into ray-project:master Mar 5, 2024
9 checks passed
@c21 c21 deleted the parallelism-test branch March 5, 2024 21:20
@scottjlee scottjlee mentioned this pull request Mar 6, 2024
Merged
8 tasks
c21 pushed a commit that referenced this pull request Mar 6, 2024
@scottjlee
[Data] Fix read_parquet_benchmark_single_node by update `parallelis…
36241dd 
…m` -> `override_num_blocks` (#43752)

Continuation of #43692, update release test code with remaining needed changes for replacing `parallelism` parameter with `override_num_blocks`.

Signed-off-by: Scott Lee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raulchen raulchen raulchen approved these changes

@ericl ericl Awaiting requested review from ericl

@scv119 scv119 Awaiting requested review from scv119

@amogkam amogkam Awaiting requested review from amogkam

@scottjlee scottjlee Awaiting requested review from scottjlee scottjlee is a code owner

@bveeramani bveeramani Awaiting requested review from bveeramani bveeramani is a code owner

@stephanie-wang stephanie-wang Awaiting requested review from stephanie-wang stephanie-wang is a code owner

@omatthew98 omatthew98 Awaiting requested review from omatthew98 omatthew98 is a code owner

Assignees
No one assigned
Labels
release-blocker P0 Issue that blocks the release
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@c21 @raulchen