-
Notifications
You must be signed in to change notification settings - Fork 5.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[serve] Make serve.shutdown()
shut down remote Serve applications
#23476
Conversation
@shrekris-anyscale there was a windows test failure in test_standalone. Retried, but can you verify if it's a real failure? |
@edoakes It is a real failure– Windows has some issues when trying to run temp files. I can either skip the test on Windows or refactor it to pass. I'll look into this. |
@edoakes I removed the context manager from the test to handle the Windows issues (see https://stackoverflow.com/a/15590253 for context), and it looks like |
@shrekris-anyscale can we update the |
Sigh... wrong PR, ignore above. Clearly reviewing while distracted today... |
Why are these changes needed?
serve.shutdown()
does not do anything if there is no cached handle to a Serve controller. If a user connects to a remote Ray cluster without runningserve.start()
, they cannot useserve.shutdown()
to shut down the remote Serve application.E.g. this does nothing, even if a Serve controller is running at
"cluster_address"
in namespace"x"
:This change makes
serve.shutdown()
first try to connect to a Serve controller before attempting to shut down. This allows a user to directly shut down a running Serve instance without potentially having to start a Serve instance withserve.start()
first.Related issue number
Closes #22254.
Checks
scripts/format.sh
to lint the changes in this PR.test_standalone2.py
.