Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] ray stop does not clean up ray_current_cluster file #45771

Closed
m-harmonic opened this issue Jun 6, 2024 · 2 comments
Closed

[Core] ray stop does not clean up ray_current_cluster file #45771

m-harmonic opened this issue Jun 6, 2024 · 2 comments
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core triage Needs triage (eg: priority, bug/not-bug, and owning component)

Comments

@m-harmonic
Copy link

What happened + What you expected to happen

  1. Start a cluster from the command line without a .yaml file:

ray start --head --port=6379 --temp-dir=/tmp/ray

  1. Stop Ray processes manually

ray stop

  1. The cluster file (ray_current_cluster) is still there and has not been cleaned up.

The problem is that starting ray again after this using ray.init() will fail because it sees the existing cluster file and tries to connect to that cluster instead of starting a new local one. Passing address="local" fixes the problem, however.

Versions / Dependencies

Ray: 2.9.3
Python: 3.10.12
OS: Ubuntu 20.04.6 LTS

Reproduction script

ray start --head --port=6379 --temp-dir=/tmp/ray
ray stop
ls /tmp/ray/ray_current_cluster

Issue Severity

Medium: It is a significant difficulty but I can work around it.

@m-harmonic m-harmonic added bug Something that is supposed to be working; but isn't triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Jun 6, 2024
@MaxVanDijck
Copy link
Contributor

Try updating your version of ray, I suspect this is the same issue that was fixed in #27666
issue: #27021

@ruisearch42 ruisearch42 added the core Issues that should be addressed in Ray Core label Jun 11, 2024
@jjyao
Copy link
Collaborator

jjyao commented Jun 18, 2024

Hi @m-harmonic, could you try the latest Ray. I think the issue is fixed.

@jjyao jjyao closed this as completed Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that is supposed to be working; but isn't core Issues that should be addressed in Ray Core triage Needs triage (eg: priority, bug/not-bug, and owning component)
Projects
None yet
Development

No branches or pull requests

4 participants