Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] Ray hangs when grpcio==1.48.0 is used. #27246

Closed
rkooo567 opened this issue Jul 29, 2022 · 1 comment
Closed

[Core] Ray hangs when grpcio==1.48.0 is used. #27246

rkooo567 opened this issue Jul 29, 2022 · 1 comment
Labels
bug Something that is supposed to be working; but isn't P1 Issue that should be fixed within a few weeks Ray 2.1
Milestone

Comments

@rkooo567
Copy link
Contributor

What happened + What you expected to happen

Not entirely sure about the root cause, but when grpcio >= 1.48.0 is used, ray hangs on starting a new process.

We are not entirely sure why this happens now, but we should figure out the root cause.

Action items

  1. We should upper bound the gRPC version because they are consistently the root cause of random hanging from core.
  2. We should figure out the root cause of 1.48.0 hanging.
  3. We should make sure whenever there's a new gRPC version release, we test accordingly.

Versions / Dependencies

Master

Reproduction script

Use grpcio==1.48.0

Issue Severity

No response

@rkooo567 rkooo567 added bug Something that is supposed to be working; but isn't P1 Issue that should be fixed within a few weeks Ray 2.1 labels Jul 29, 2022
@rkooo567 rkooo567 added this to the Core Backlog milestone Jul 29, 2022
@jjyao
Copy link
Collaborator

jjyao commented Aug 8, 2022

Duplicate of #27299

@jjyao jjyao marked this as a duplicate of #27299 Aug 8, 2022
@jjyao jjyao closed this as completed Aug 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't P1 Issue that should be fixed within a few weeks Ray 2.1
Projects
None yet
Development

No branches or pull requests

2 participants