Allow control of some serve configuration via env vars #47533
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Tim Paine <[email protected]>
timkpaine
force-pushed
the
tkp/serveenvvar
branch
from
8d168d4
to
2299af7
Compare
timkpaine
commented
Sep 6, 2024
|
|
||
| #: HTTP Port | ||
| DEFAULT_HTTP_PORT = 8000 | ||
| DEFAULT_HTTP_PORT = int(os.environ.get("RAY_SERVE_DEFAULT_HTTP_PORT", 8000)) | ||
|
|
||
| #: Uvicorn timeout_keep_alive Config | ||
| DEFAULT_UVICORN_KEEP_ALIVE_TIMEOUT_S = 5 |
There was a problem hiding this comment.
there are more here and below, happy to expose all of them if it makes sense to do so
There was a problem hiding this comment.
we can leave the surface area minimal for now
|
Lgtm! |
edoakes
approved these changes
Sep 10, 2024
Comment on lines
+19
to
+21
| DEFAULT_HTTP_ADDRESS = os.environ.get( | ||
| "RAY_SERVE_DEFAULT_HTTP_ADDRESS", "http://127.0.0.1:8000" | ||
| ) |
There was a problem hiding this comment.
can you check what this one is used for and why it's different from the two below?
There was a problem hiding this comment.
@GeneDer can you follow up on this one? going to merge the PR
There was a problem hiding this comment.
Oh interesting this one is actually not used anywhere. Let me follow up to just remove it
|
|
||
| #: HTTP Port | ||
| DEFAULT_HTTP_PORT = 8000 | ||
| DEFAULT_HTTP_PORT = int(os.environ.get("RAY_SERVE_DEFAULT_HTTP_PORT", 8000)) | ||
|
|
||
| #: Uvicorn timeout_keep_alive Config | ||
| DEFAULT_UVICORN_KEEP_ALIVE_TIMEOUT_S = 5 |
There was a problem hiding this comment.
we can leave the surface area minimal for now
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Oct 15, 2024
…7533) When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the [ray-cluster helm chart](https://github.com/ray-project/kuberay/blob/master/helm-chart/ray-cluster/values.yaml) there is no ability to set the default serve arguments). This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient. Signed-off-by: Tim Paine <[email protected]> Signed-off-by: ujjawal-khare <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
When a serve app is launched, serve will startup automatically. In certain places like k8s, it can be difficult to preconfigure serve (e.g. in the ray-cluster helm chart there is no ability to set the default serve arguments).
This means you need to either be explicit when you start serve, or if it starts up automatically you may need to shut it down, then restart it, which is inconvenient.
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.