diff --git a/apiserver/README.md b/apiserver/README.md index 0f865476ba..46a4b2bb35 100644 --- a/apiserver/README.md +++ b/apiserver/README.md @@ -2,21 +2,20 @@ The KubeRay APIServer provides gRPC and HTTP APIs to manage KubeRay resources. -**Note** +## Note - The KubeRay APIServer is an optional component. It provides a layer of simplified - configuration for KubeRay resources. The KubeRay API server is used internally - by some organizations to back user interfaces for KubeRay resource management. +```text +The KubeRay APIServer is an optional component. It provides a layer of simplified configuration for KubeRay resources. The KubeRay API server is used internally by some organizations to back user interfaces for KubeRay resource management. - The KubeRay APIServer is community-managed and is not officially endorsed by the - Ray maintainers. At this time, the only officially supported methods for - managing KubeRay resources are +The KubeRay APIServer is community-managed and is not officially endorsed by the Ray maintainers. At this time, the only officially supported methods for +managing KubeRay resources are: - - Direct management of KubeRay custom resources via kubectl, kustomize, and Kubernetes language clients. - - Helm charts. +- Direct management of KubeRay custom resources via kubectl, kustomize, and Kubernetes language clients. +- Helm charts. - KubeRay APIServer maintainer contacts (GitHub handles): - @Jeffwan @scarlet25151 +KubeRay APIServer maintainer contacts (GitHub handles): +@Jeffwan @scarlet25151 +``` ## Installation @@ -28,39 +27,75 @@ Make sure the version of Helm is v3+. Currently, [existing CI tests](https://git helm version ``` +### Install KubeRay Operator + +* Install a stable version via Helm repository (only supports KubeRay v0.4.0+) + + ```sh + # Install the KubeRay helm repo + helm repo add kuberay https://ray-project.github.io/kuberay-helm/ + + # Install KubeRay Operator v0.6.0. + helm install kuberay-operator kuberay/kuberay-operator --version 0.6.0 + + # Check the KubeRay Operator Pod in `default` namespace + kubectl get pods + # NAME READY STATUS RESTARTS AGE + # kuberay-operator-7456c6b69b-t6pt7 1/1 Running 0 172m + + ``` + ### Install KubeRay APIServer * Install a stable version via Helm repository (only supports KubeRay v0.4.0+) + ```sh + # Install the KubeRay helm repo helm repo add kuberay https://ray-project.github.io/kuberay-helm/ # Install KubeRay APIServer v0.6.0. - helm install kuberay-apiserver kuberay/kuberay-apiserver --version 0.6.0 + helm install -n ray-system kuberay-apiserver kuberay/kuberay-apiserver --version 0.6.0 # Check the KubeRay APIServer Pod in `default` namespace kubectl get pods - # NAME READY STATUS RESTARTS AGE - # kuberay-apiserver-67b46b88bf-m7dzg 1/1 Running 0 6s + # NAME READY STATUS RESTARTS AGE + # kuberay-apiserver-857869f665-b94px 1/1 Running 0 86m + # kuberay-operator-7456c6b69b-t6pt7 1/1 Running 0 172m ``` * Install the nightly version + ```sh # Step1: Clone KubeRay repository # Step2: Move to `helm-chart/kuberay-apiserver` + cd helm-chart/kuberay-apiserver # Step3: Install KubeRay APIServer helm install kuberay-apiserver . ``` +* Install the current (working branch) version + + ```sh + # Step1: Clone KubeRay repository + + # Step2: Change directory to the api server + cd apiserver + + # Step3: Build docker image, create a local kind cluster and deploy api server (using helm) + make docker-image cluster load-image deploy + ``` + ### List the chart -To list the `my-release` deployment: +To list the deployments: ```sh helm ls -# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION -# kuberay-apiserver default 1 2023-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx deployed kuberay-apiserver-0.6.0 +# NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION +# kuberay-apiserver default 1 2023-09-15 11:34:33.895054 +0300 EEST deployed #kuberay-apiserver-0.6.0 +# kuberay-operator default 1 2023-09-15 10:08:44.637539 +0300 EEST deployed kuberay-operator-0.6.0 ``` ### Uninstall the Chart @@ -78,33 +113,32 @@ kubectl get pods After the deployment we may use the `{{baseUrl}}` to access the -- (default) for nodeport access, we provide the default http port `31888` for connection and you can connect it using. +* (default) for nodeport access, we provide the default http port `31888` for connection and you can connect to the server using it -- for ingress access, you will need to create your own ingress +* for ingress access, you will need to create your own ingress The requests parameters detail can be seen in [KubeRay swagger](https://github.com/ray-project/kuberay/tree/master/proto/swagger), here we only present some basic example: -### Setup end-to-end test - -0. (Optional) You may use your local kind cluster or minikube - -```bash -cat <:31888/swagger-ui` -## Full definition of payload +## Full definition endpoints ### Compute Template -For the purpose to simplify the setting of resource, we abstract the resource -of the pods template resource to the `compute template` for usage, you can -define the resource in the `compute template` and then choose the appropriate -template for your `head` and `workergroup` when you are creating the real objects of `RayCluster`, `RayJobs` or `RayService`. +For the purpose to simplify the setting of resource, we abstract the resource of the pods template resource to the `compute template` for usage, you can define the resource in the `compute template` and then choose the appropriate template for your `head` and `workergroup` when you are creating the real objects of `RayCluster`, `RayJobs` or `RayService`. #### Create compute templates in a given namespace -``` +```text POST {{baseUrl}}/apis/v1alpha2/namespaces//compute_templates ``` -```json -{ - "name": "default-template", - "namespace": "", - "cpu": 2, - "memory": 4, - "gpu": 1, - "gpuAccelerator": "Tesla-V100" -} -``` +Examples: + +* Request + + ```sh + curl --silent -X 'POST' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/compute_templates' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "name": "default-template", + "namespace": "ray-system", + "cpu": 2, + "memory": 4 + }' + ``` + +* Response + + ```json + { + "name": "default-template", + "namespace": "ray-system", + "cpu": 2, + "memory": 4 + } + ``` #### List all compute templates in a given namespace -``` +```text GET {{baseUrl}}/apis/v1alpha2/namespaces//compute_templates ``` -```json -{ - "compute_templates": [ - { - "name": "default-template", - "namespace": "", - "cpu": 2, - "memory": 4, - "gpu": 1, - "gpu_accelerator": "Tesla-V100" - } +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/compute_templates' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + { + "computeTemplates": [ + { + "name": "default-template", + "namespace": "ray-system", + "cpu": 2, + "memory": 4 + } ] -} -``` + } + ``` #### List all compute templates in all namespaces -``` +```text GET {{baseUrl}}/apis/v1alpha2/compute_templates ``` +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/compute_templates' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + { + "computeTemplates": [ + { + "name": "default-template", + "namespace": "default", + "cpu": 2, + "memory": 4 + }, + { + "name": "default-template", + "namespace": "ray-system", + "cpu": 2, + "memory": 4 + } + ] + } + ``` + #### Get compute template by name -``` +```text GET {{baseUrl}}/apis/v1alpha2/namespaces//compute_templates/ ``` -#### Delete compute template by name - -``` -DELETE {{baseUrl}}/apis/v1alpha2/namespaces//compute_templates/ -``` +Examples: -### Clusters +* Request -#### Create cluster in a given namespace + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/compute_templates/default-template' \ + -H 'accept: application/json' + ``` -``` -POST {{baseUrl}}/apis/v1alpha2/namespaces//clusters -``` +* Response: -payload -```json -{ - "name": "test-cluster", - "namespace": "", - "user": "jiaxin.shan", - "version": "1.9.2", - "environment": "DEV", - "clusterSpec": { - "headGroupSpec": { - "computeTemplate": "head-template", - "image": "ray.io/ray:1.9.2", - "serviceType": "NodePort", - "rayStartParams": {} - }, - "workerGroupSpec": [ - { - "groupName": "small-wg", - "computeTemplate": "worker-template", - "image": "ray.io/ray:1.9.2", - "replicas": 2, - "minReplicas": 0, - "maxReplicas": 5, - "rayStartParams": {} - } - ] + ```json + { + "name": "default-template", + "namespace": "ray-system", + "cpu": 2, + "memory": 4 } -} -``` + ``` -#### List all clusters in a given namespace +#### Delete compute template by name -``` -GET {{baseUrl}}/apis/v1alpha2/namespaces//clusters +```text +DELETE {{baseUrl}}/apis/v1alpha2/namespaces//compute_templates/ ``` -```json -{ - "clusters": [ - { - "name": "test-cluster", - "namespace": "", - "user": "jiaxin.shan", - "version": "1.9.2", - "environment": "DEV", - "cluster_spec": { - "head_group_spec": { - "compute_template": "head-template", - "image": "rayproject/ray:1.9.2", - "service_type": "NodePort", - "ray_start_params": { - "dashboard-host": "0.0.0.0", - "node-ip-address": "$MY_POD_IP", - "port": "6379" - } - }, - "worker_group_spec": [ - { - "group_name": "small-wg", - "compute_template": "worker-template", - "image": "rayproject/ray:1.9.2", - "replicas": 2, - "min_replicas": 0, - "max_replicas": 5, - "ray_start_params": { - "node-ip-address": "$MY_POD_IP", - } - } - ] - }, - "created_at": "2022-03-13T15:13:09Z", - "deleted_at": null - }, - ] -} -``` +Examples: -#### List all clusters in all namespaces +* Request -``` -GET {{baseUrl}}/apis/v1alpha2/clusters -``` + ```sh + curl --silent -X 'DELETE' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/compute_templates/default-template' \ + -H 'accept: application/json' + ``` -#### Get cluster by its name and namespace +* Response -``` -GET {{baseUrl}}/apis/v1alpha2/namespaces//clusters/ -``` + ```json + {} + ``` +### Clusters -#### Delete cluster by its name and namespace +#### Create cluster in a given namespace -``` -DELETE {{baseUrl}}/apis/v1alpha2/namespaces//clusters/ +```text +POST {{baseUrl}}/apis/v1alpha2/namespaces//clusters ``` -### RayJob +Examples: -#### Create ray job in a given namespace +* Request -``` -POST {{baseUrl}}/apis/v1alpha2/namespaces//jobs -``` + ```sh + curl --silent -X 'POST' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/clusters' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "name": "test-cluster", + "namespace": "ray-system", + "user": "3cpo", + "version": "2.6.3", + "environment": "DEV", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + }, + "volumes": [] + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 2, + "minReplicas": 0, + "maxReplicas": 5, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + } + }' + ``` +* Response -payload -```json -{ - "name": "string", - "namespace": "string", - "user": "string", - "entrypoint": "string", - "metadata": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "runtimeEnv": "string", - "jobId": "string", - "shutdownAfterJobFinishes": true, - "clusterSelector": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "clusterSpec": { - "headGroupSpec": { - "computeTemplate": "string", - "image": "string", - "serviceType": "string", - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + ```json + { + "name": "test-cluster", + "namespace": "ray-system", + "user": "3cpo", + "version": "2.6.3", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } }, - "volumes": [ + "workerGroupSpec": [ { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 2, + "minReplicas": 5, + "maxReplicas": 2, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } } ] }, - "workerGroupSpec": [ - { - "groupName": "string", - "computeTemplate": "string", - "image": "string", - "replicas": 0, - "minReplicas": 0, - "maxReplicas": 0, - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "volumes": [ - { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" - } - ] - } - ] - }, - "ttlSecondsAfterFinished": 0, - "createdAt": "2022-08-19T21:20:30.494Z", - "deleteAt": "2022-08-19T21:20:30.494Z", - "jobStatus": "string", - "jobDeploymentStatus": "string", - "message": "string" -} -``` + "annotations": { + "ray.io/creation-timestamp": "2023-09-14 12:31:58.070884 +0000 UTC" + }, + "createdAt": "2023-09-14T12:31:58Z" + } + ``` -#### List all jobs in a given namespace +#### List all clusters in a given namespace -``` -GET {{baseUrl}}/apis/v1alpha2/namespaces//jobs +```text +GET {{baseUrl}}/apis/v1alpha2/namespaces//clusters ``` -Response -```json -{ - "jobs": [ - { - "name": "string", - "namespace": "string", - "user": "string", - "entrypoint": "string", - "metadata": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "runtimeEnv": "string", - "jobId": "string", - "shutdownAfterJobFinishes": true, - "clusterSelector": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "clusterSpec": { - "headGroupSpec": { - "computeTemplate": "string", - "image": "string", - "serviceType": "string", - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/clusters' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + { + "clusters": [ + { + "name": "test-cluster", + "namespace": "ray-system", + "user": "3cpo", + "version": "2.6.3", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } }, - "volumes": [ + "workerGroupSpec": [ { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 2, + "minReplicas": 5, + "maxReplicas": 2, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } } ] }, - "workerGroupSpec": [ + "annotations": { + "ray.io/creation-timestamp": "2023-09-15 11:31:08.304211723 +0000 UTC" + }, + "createdAt": "2023-09-15T11:31:08Z", + "clusterState": "ready", + "events": [ { - "groupName": "string", - "computeTemplate": "string", - "image": "string", - "replicas": 0, - "minReplicas": 0, - "maxReplicas": 0, - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "volumes": [ - { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" - } - ] + "id": "test-cluster.17850f20cdb520b5", + "name": "test-cluster-test-cluster.17850f20cdb520b5", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created service test-cluster-head-svc", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17850f20d04c4601", + "name": "test-cluster-test-cluster.17850f20d04c4601", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created head pod test-cluster-head-v4dmh", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17850f20d642aa31", + "name": "test-cluster-test-cluster.17850f20d642aa31", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 2 } - ] - }, - "ttlSecondsAfterFinished": 0, - "createdAt": "2022-08-19T21:31:24.352Z", - "deleteAt": "2022-08-19T21:31:24.352Z", - "jobStatus": "string", - "jobDeploymentStatus": "string", - "message": "string" - } - ] -} -``` - -#### List all jobs in all namespaces - -``` -GET {{baseUrl}}/apis/v1alpha2/jobs -``` + ], + "serviceEndpoint": { + "dashboard": "30529", + "head": "32208", + "metrics": "32623", + "redis": "30105" + } + } + ] + } + ``` -#### Get job by its name and namespace +#### List all clusters in all namespaces +```text +GET {{baseUrl}}/apis/v1alpha2/clusters ``` -GET {{baseUrl}}/apis/v1alpha2/namespaces//jobs/ -``` - -#### Delete job by its name and namespace +Examples: -``` -DELETE {{baseUrl}}/apis/v1alpha2/namespaces//jobs/ -``` +* Request + ```sh + curl -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/clusters' \ + -H 'accept: application/json' + ``` -### RayService +* Response -#### Create ray service in a given namespace - -``` -POST {{baseUrl}}/apis/v1alpha2/namespaces//services -``` - -payload - -```json -{ - "name": "string", - "namespace": "string", - "user": "string", - "serveDeploymentGraphSpec": { - "importPath": "string", - "runtimeEnv": "string", - "serveConfigs": [ + ```json + { + "clusters": [ { - "deploymentName": "string", - "replicas": 0, - "routePrefix": "string", - "maxConcurrentQueries": 0, - "userConfig": "string", - "autoscalingConfig": "string", - "actorOptions": { - "runtimeEnv": "string", - "cpus": 0, - "gpu": 0, - "memory": 0, - "objectStoreMemory": 0, - "resource": "string", - "accceleratorType": "string" + "name": "test-cluster", + "namespace": "ray-system", + "user": "3cpo", + "version": "2.6.3", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 2, + "minReplicas": 5, + "maxReplicas": 2, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "annotations": { + "ray.io/creation-timestamp": "2023-09-15 11:31:08.304211723 +0000 UTC" + }, + "createdAt": "2023-09-15T11:31:08Z", + "clusterState": "ready", + "events": [ + { + "id": "test-cluster.17850f20cdb520b5", + "name": "test-cluster-test-cluster.17850f20cdb520b5", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created service test-cluster-head-svc", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17850f20d04c4601", + "name": "test-cluster-test-cluster.17850f20d04c4601", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created head pod test-cluster-head-v4dmh", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17850f20d642aa31", + "name": "test-cluster-test-cluster.17850f20d642aa31", + "createdAt": "2023-09-15T11:31:08Z", + "firstTimestamp": "2023-09-15T11:31:08Z", + "lastTimestamp": "2023-09-15T11:31:08Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 2 + } + ], + "serviceEndpoint": { + "dashboard": "30529", + "head": "32208", + "metrics": "32623", + "redis": "30105" } } ] - }, - "clusterSpec": { - "headGroupSpec": { - "computeTemplate": "string", - "image": "string", - "serviceType": "string", - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + } + ``` + +#### Get cluster by its name and namespace + +```text +GET {{baseUrl}}/apis/v1alpha2/namespaces//clusters/ +``` + +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/clusters' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + { + "name": "test-cluster", + "namespace": "ray-system", + "user": "3cpo", + "version": "2.6.3", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } }, - "volumes": [ + "workerGroupSpec": [ { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 2, + "minReplicas": 5, + "maxReplicas": 2, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } } ] }, - "workerGroupSpec": [ + "createdAt": "2023-09-06T11:42:03Z", + "events": [ { - "groupName": "string", - "computeTemplate": "string", - "image": "string", - "replicas": 0, - "minReplicas": 0, - "maxReplicas": 0, - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" - }, - "volumes": [ - { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" - } - ] - } - ] - }, - "rayServiceStatus": { - "applicationStatus": "string", - "applicationMessage": "string", - "serveDeploymentStatus": [ + "id": "test-cluster.17824c260cc502ff", + "name": "test-cluster-test-cluster.17824c260cc502ff", + "createdAt": "2023-09-06T11:35:36Z", + "firstTimestamp": "2023-09-06T11:35:36Z", + "lastTimestamp": "2023-09-06T11:35:36Z", + "reason": "Created", + "message": "Created service test-cluster-head-svc", + "type": "Normal", + "count": 1 + }, { - "deploymentName": "string", - "status": "string", - "message": "string" - } - ], - "rayServiceEvent": [ + "id": "test-cluster.17824c26117bd44f", + "name": "test-cluster-test-cluster.17824c26117bd44f", + "createdAt": "2023-09-06T11:35:36Z", + "firstTimestamp": "2023-09-06T11:35:36Z", + "lastTimestamp": "2023-09-06T11:35:36Z", + "reason": "Created", + "message": "Created head pod test-cluster-head-bh75l", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17824c261783b274", + "name": "test-cluster-test-cluster.17824c261783b274", + "createdAt": "2023-09-06T11:35:36Z", + "firstTimestamp": "2023-09-06T11:35:36Z", + "lastTimestamp": "2023-09-06T11:35:36Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 2 + }, + { + "id": "test-cluster.17824c8032b97213", + "name": "test-cluster-test-cluster.17824c8032b97213", + "createdAt": "2023-09-06T11:42:03Z", + "firstTimestamp": "2023-09-06T11:42:03Z", + "lastTimestamp": "2023-09-06T11:42:03Z", + "reason": "Created", + "message": "Created service test-cluster-head-svc", + "type": "Normal", + "count": 1 + }, { - "id": "string", - "name": "string", - "createdAt": "2022-08-19T21:30:01.097Z", - "firstTimestamp": "2022-08-19T21:30:01.097Z", - "lastTimestamp": "2022-08-19T21:30:01.097Z", - "reason": "string", - "message": "string", - "type": "string", - "count": 0 + "id": "test-cluster.17824c8033d66898", + "name": "test-cluster-test-cluster.17824c8033d66898", + "createdAt": "2023-09-06T11:42:03Z", + "firstTimestamp": "2023-09-06T11:42:03Z", + "lastTimestamp": "2023-09-06T11:42:03Z", + "reason": "Created", + "message": "Created head pod test-cluster-head-m4nng", + "type": "Normal", + "count": 1 + }, + { + "id": "test-cluster.17824c803897ec83", + "name": "test-cluster-test-cluster.17824c803897ec83", + "createdAt": "2023-09-06T11:42:03Z", + "firstTimestamp": "2023-09-06T11:42:03Z", + "lastTimestamp": "2023-09-06T11:42:03Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 2 } ], - "rayClusterName": "string", - "rayClusterState": "string", "serviceEndpoint": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + "dashboard": "32348", + "head": "32349", + "metrics": "32134", + "redis": "30092" } - }, - "createdAt": "2022-08-19T21:30:01.097Z", - "deleteAt": "2022-08-19T21:30:01.097Z" -} + } + ``` + +#### Delete cluster by its name and namespace + +```text +DELETE {{baseUrl}}/apis/v1alpha2/namespaces//clusters/ ``` -#### List all services in a given namespace +Examples: + +* Request + + ```sh + curl --silent -X 'DELETE' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/clusters/test-cluster' \ + -H 'accept: application/json' + ``` + +* Response: + + ```json + {} + ``` + +### RayJob + +#### Create ray job in a given namespace +```text +POST {{baseUrl}}/apis/v1alpha2/namespaces//jobs ``` -GET {{baseUrl}}/apis/v1alpha2/namespaces//services + +Examples: + +* Request + + ```sh + curl --silent -X 'POST' \ + 'http://localhost:31888/apis/v1alpha2/jobs?namespace=ray-system' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "name": "rayjob-test", + "namespace": "ray-system", + "user": "3cp0", + "entrypoint": "python -V", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 1, + "minReplicas": 0, + "maxReplicas": 1, + "rayStartParams": { + "metrics-export-port": "8080" + } + } + ] + } + } + ' + ``` + +* Response + + ```json + { + "name": "rayjob-test", + "namespace": "ray-system", + "user": "3cp0", + "entrypoint": "python -V", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 1, + "minReplicas": 1, + "maxReplicas": 1, + "rayStartParams": { + "metrics-export-port": "8080" + } + } + ] + }, + "createdAt": "2023-09-18T07:13:34Z" + } + ``` + +#### List all jobs in a given namespace + +```text +GET {{baseUrl}}/apis/v1alpha2/namespaces//jobs ``` -Response -```json +Examples: + +* Request - "name": "string", - "namespace": "string", - "user": "string", - "serveDeploymentGraphSpec": { - "importPath": "string", - "runtimeEnv": "string", - "serveConfigs": [ + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/jobs' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + { + "jobs": [ { - "deploymentName": "string", - "replicas": 0, - "routePrefix": "string", - "maxConcurrentQueries": 0, - "userConfig": "string", - "autoscalingConfig": "string", - "actorOptions": { - "runtimeEnv": "string", - "cpus": 0, - "gpu": 0, - "memory": 0, - "objectStoreMemory": 0, - "resource": "string", - "accceleratorType": "string" - } + "name": "rayjob-test", + "namespace": "ray-system", + "user": "3cp0", + "entrypoint": "python -V", + "jobId": "rayjob-test-k58tz", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 1, + "minReplicas": 1, + "maxReplicas": 1, + "rayStartParams": { + "metrics-export-port": "8080" + } + } + ] + }, + "createdAt": "2023-09-18T07:13:34Z", + "jobStatus": "SUCCEEDED", + "jobDeploymentStatus": "Running", + "message": "Job finished successfully." } ] - }, - "clusterSpec": { - "headGroupSpec": { - "computeTemplate": "string", - "image": "string", - "serviceType": "string", - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + } + ``` + +#### List all jobs in all namespaces + +```text +GET {{baseUrl}}/apis/v1alpha2/jobs +``` + +Examples: + +* Request: + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/jobs' \ + -H 'accept: application/json' + + ``` + +* Response + + ```json + { + "jobs": [ + { + "name": "rayjob-test", + "namespace": "ray-system", + "user": "3cp0", + "entrypoint": "python -V", + "jobId": "rayjob-test-k58tz", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 1, + "minReplicas": 1, + "maxReplicas": 1, + "rayStartParams": { + "metrics-export-port": "8080" + } + } + ] + }, + "createdAt": "2023-09-18T07:13:34Z", + "jobStatus": "SUCCEEDED", + "jobDeploymentStatus": "Running", + "message": "Job finished successfully." + } + ] + } + ``` + +#### Get job by its name and namespace + +```text +GET {{baseUrl}}/apis/v1alpha2/namespaces//jobs/ +``` + +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/jobs/rayjob-test' \ + -H 'accept: application/json' + ``` + +* Response: + + ```json + { + "name": "rayjob-test", + "namespace": "ray-system", + "user": "3cp0", + "entrypoint": "python -V", + "jobId": "rayjob-test-k58tz", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0" + } }, - "volumes": [ + "workerGroupSpec": [ { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3", + "replicas": 1, + "minReplicas": 1, + "maxReplicas": 1, + "rayStartParams": { + "metrics-export-port": "8080" + } } ] }, - "workerGroupSpec": [ - { - "groupName": "string", - "computeTemplate": "string", - "image": "string", - "replicas": 0, - "minReplicas": 0, - "maxReplicas": 0, - "rayStartParams": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + "createdAt": "2023-09-18T07:13:34Z", + "jobStatus": "SUCCEEDED", + "jobDeploymentStatus": "Running", + "message": "Job finished successfully." + } + ``` + +#### Delete job by its name and namespace + +```text +DELETE {{baseUrl}}/apis/v1alpha2/namespaces//jobs/ +``` + +Examples: + +* Request + + ```sh + curl --silent -X 'DELETE' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/jobs/rayjob-test' \ + -H 'accept: application/json' + ``` + +* Response + + ```json + {} + ``` + +### RayService + +#### Create ray service in a given namespace + +```text +POST {{baseUrl}}/apis/v1alpha2/namespaces//services +``` + +Examples: + +* Request (V1) + + ```sh + curl --silent -X 'POST' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/services' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d '{ + "name": "test-v1", + "namespace": "ray-system", + "user": "user", + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "serveDeploymentGraphSpec": { + "importPath": "fruit.deployment_graph", + "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"\n", + "serveConfigs": [ + { + "deploymentName": "OrangeStand", + "replicas": 1, + "userConfig": "price: 2", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "PearStand", + "replicas": 1, + "userConfig": "price: 1", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "FruitMarket", + "replicas": 1, + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "DAGDriver", + "replicas": 1, + "routePrefix": "/", + "actorOptions": { + "cpusPerActor": 0.1 + } + } + ] + }, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + }, + "volumes": [] }, - "volumes": [ + "workerGroupSpec": [ { - "mountPath": "string", - "volumeType": "PERSISTENT_VOLUME_CLAIM", - "name": "string", - "source": "string", - "readOnly": true, - "hostPathType": "DIRECTORY", - "mountPropagationMode": "NONE" + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "replicas": 1, + "minReplicas": 0, + "maxReplicas": 5, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } } ] } - ] - }, - "rayServiceStatus": { - "applicationStatus": "string", - "applicationMessage": "string", - "serveDeploymentStatus": [ - { - "deploymentName": "string", - "status": "string", - "message": "string" - } - ], - "rayServiceEvent": [ - { - "id": "string", - "name": "string", - "createdAt": "2022-08-19T21:33:15.485Z", - "firstTimestamp": "2022-08-19T21:33:15.485Z", - "lastTimestamp": "2022-08-19T21:33:15.485Z", - "reason": "string", - "message": "string", - "type": "string", - "count": 0 - } - ], - "rayClusterName": "string", - "rayClusterState": "string", - "serviceEndpoint": { - "additionalProp1": "string", - "additionalProp2": "string", - "additionalProp3": "string" + }' + ``` + +* Response (V1) + + ```json + { + "name": "test-v1", + "namespace": "ray-system", + "user": "user", + "serveDeploymentGraphSpec": { + "importPath": "fruit.deployment_graph", + "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"\n", + "serveConfigs": [ + { + "deploymentName": "OrangeStand", + "replicas": 1, + "userConfig": "price: 2", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "PearStand", + "replicas": 1, + "userConfig": "price: 1", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "FruitMarket", + "replicas": 1, + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "DAGDriver", + "replicas": 1, + "routePrefix": "/", + "actorOptions": { + "cpusPerActor": 0.1 + } + } + ] + }, + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "replicas": 1, + "minReplicas": 5, + "maxReplicas": 1, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "rayServiceStatus": {}, + "createdAt": "2023-09-18T07:55:23Z", + "deleteAt": "1969-12-31T23:59:59Z" + } + ``` + +* Request (V2) + + ```sh + curl -X 'POST' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/services' \ + -H 'accept: application/json' \ + -H 'Content-Type: application/json' \ + -d ' + { + "name": "test-v2", + "namespace": "ray-system", + "user": "user", + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "serveConfigV2": "applications:\n - name: fruit_app\n import_path: fruit.deployment_graph\n route_prefix: /fruit\n runtime_env:\n working_dir: \"https://github.com/ray-project/test_dag/archive/41d09119cbdf8450599f993f51318e9e27c59098.zip\"\n deployments:\n - name: MangoStand\n num_replicas: 1\n user_config:\n price: 3\n ray_actor_options:\n num_cpus: 0.1\n - name: OrangeStand\n num_replicas: 1\n user_config:\n price: 2\n ray_actor_options:\n num_cpus: 0.1\n - name: PearStand\n num_replicas: 1\n user_config:\n price: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: FruitMarket\n num_replicas: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: DAGDriver\n num_replicas: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: math_app\n import_path: conditional_dag.serve_dag\n route_prefix: /calc\n runtime_env:\n working_dir: \"https://github.com/ray-project/test_dag/archive/41d09119cbdf8450599f993f51318e9e27c59098.zip\"\n deployments:\n - name: Adder\n num_replicas: 1\n user_config:\n increment: 3\n ray_actor_options:\n num_cpus: 0.1\n - name: Multiplier\n num_replicas: 1\n user_config:\n factor: 5\n ray_actor_options:\n num_cpus: 0.1\n - name: Router\n num_replicas: 1\n - name: create_order\n num_replicas: 1\n - name: DAGDriver\n num_replicas: 1\n", + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + }, + "volumes": [] + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "replicas": 1, + "minReplicas": 0, + "maxReplicas": 5, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] } - }, - "createdAt": "2022-08-19T21:33:15.485Z", - "deleteAt": "2022-08-19T21:33:15.485Z" -} + }' + ``` + +* Response (V2) + + ```json + { + "name": "test-v2", + "namespace": "ray-system", + "user": "user", + "serveConfigV2": "applications:\n - name: fruit_app\n import_path: fruit.deployment_graph\n route_prefix: /fruit\n runtime_env:\n working_dir: \"https://github.com/ray-project/test_dag/archive/41d09119cbdf8450599f993f51318e9e27c59098.zip\"\n deployments:\n - name: MangoStand\n num_replicas: 1\n user_config:\n price: 3\n ray_actor_options:\n num_cpus: 0.1\n - name: OrangeStand\n num_replicas: 1\n user_config:\n price: 2\n ray_actor_options:\n num_cpus: 0.1\n - name: PearStand\n num_replicas: 1\n user_config:\n price: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: FruitMarket\n num_replicas: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: DAGDriver\n num_replicas: 1\n ray_actor_options:\n num_cpus: 0.1\n - name: math_app\n import_path: conditional_dag.serve_dag\n route_prefix: /calc\n runtime_env:\n working_dir: \"https://github.com/ray-project/test_dag/archive/41d09119cbdf8450599f993f51318e9e27c59098.zip\"\n deployments:\n - name: Adder\n num_replicas: 1\n user_config:\n increment: 3\n ray_actor_options:\n num_cpus: 0.1\n - name: Multiplier\n num_replicas: 1\n user_config:\n factor: 5\n ray_actor_options:\n num_cpus: 0.1\n - name: Router\n num_replicas: 1\n - name: create_order\n num_replicas: 1\n - name: DAGDriver\n num_replicas: 1\n", + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "replicas": 1, + "minReplicas": 5, + "maxReplicas": 1, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "rayServiceStatus": { + "rayServiceEvents": [ + { + "id": "test-v2.1785f3479cd90185", + "name": "test-v2-test-v2.1785f3479cd90185", + "createdAt": "2023-09-18T09:12:03Z", + "firstTimestamp": "2023-09-18T09:12:03Z", + "lastTimestamp": "2023-09-18T09:12:35Z", + "reason": "ServiceUnhealthy", + "message": "The service is in an unhealthy state. Controller will perform a round of actions in 10s.", + "type": "Normal", + "count": 5 + } + ] + }, + "createdAt": "2023-09-18T09:13:24Z", + "deleteAt": "1969-12-31T23:59:59Z" + } + ``` + +#### List all services in a given namespace + +```text +GET {{baseUrl}}/apis/v1alpha2/namespaces//services ``` +Examples + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/services' \ + -H 'accept: application/json' + ``` + +* Response: + + ```json + { + "services": [ + { + "name": "test-v1", + "namespace": "ray-system", + "user": "user", + "serveDeploymentGraphSpec": { + "importPath": "fruit.deployment_graph", + "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"\n", + "serveConfigs": [ + { + "deploymentName": "OrangeStand", + "replicas": 1, + "userConfig": "price: 2", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "PearStand", + "replicas": 1, + "userConfig": "price: 1", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "FruitMarket", + "replicas": 1, + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "DAGDriver", + "replicas": 1, + "routePrefix": "/", + "actorOptions": { + "cpusPerActor": 0.1 + } + } + ] + }, + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310", + "replicas": 1, + "minReplicas": 5, + "maxReplicas": 1, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "rayServiceStatus": { + "rayServiceEvents": [ + { + "id": "test-v1.1785ef1932a53822", + "name": "test-v1-test-v1.1785ef1932a53822", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "ServiceUnhealthy", + "message": "The service is in an unhealthy state. Controller will perform a round of actions in 10s.", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1962bfb4c2", + "name": "test-v1-test-v1.1785ef1962bfb4c2", + "createdAt": "2023-09-18T07:55:27Z", + "firstTimestamp": "2023-09-18T07:55:27Z", + "lastTimestamp": "2023-09-18T07:55:30Z", + "reason": "WaitForServeDeploymentReady", + "message": "Put \"http://test-v1-raycluster-vcvz2-head-svc.ray-system.svc.cluster.local:52365/api/serve/deployments/\": dial tcp 10.96.98.134:52365: connect: connection refused", + "type": "Normal", + "count": 3 + }, + { + "id": "test-v1.1785ef1b0f379aa9", + "name": "test-v1-test-v1.1785ef1b0f379aa9", + "createdAt": "2023-09-18T07:55:34Z", + "firstTimestamp": "2023-09-18T07:55:34Z", + "lastTimestamp": "2023-09-18T07:55:34Z", + "reason": "SubmittedServeDeployment", + "message": "Controller sent API request to update Serve deployments on cluster test-v1-raycluster-vcvz2", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1bbaa5aa5f", + "name": "test-v1-test-v1.1785ef1bbaa5aa5f", + "createdAt": "2023-09-18T07:55:37Z", + "firstTimestamp": "2023-09-18T07:55:37Z", + "lastTimestamp": "2023-09-18T07:55:39Z", + "reason": "ServiceNotReady", + "message": "The service is not ready yet. Controller will perform a round of actions in 2s.", + "type": "Normal", + "count": 2 + }, + { + "id": "test-v1.1785ef1cd4c74326", + "name": "test-v1-test-v1.1785ef1cd4c74326", + "createdAt": "2023-09-18T07:55:41Z", + "firstTimestamp": "2023-09-18T07:55:41Z", + "lastTimestamp": "2023-09-18T08:00:27Z", + "reason": "Running", + "message": "The Serve applicaton is now running and healthy.", + "type": "Normal", + "count": 136 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1920827324", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1920827324", + "createdAt": "2023-09-18T07:55:25Z", + "firstTimestamp": "2023-09-18T07:55:25Z", + "lastTimestamp": "2023-09-18T07:55:25Z", + "reason": "Created", + "message": "Created service test-v1-raycluster-vcvz2-head-svc", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1926720c3b", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1926720c3b", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created head pod test-v1-raycluster-vcvz2-head-959rs", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 1 + } + ], + "rayClusterName": "test-v1-raycluster-vcvz2", + "serveApplicationStatus": [ + { + "name": "default", + "status": "RUNNING", + "serveDeploymentStatus": [ + { + "deploymentName": "default_DAGDriver", + "status": "HEALTHY" + }, + { + "deploymentName": "default_FruitMarket", + "status": "HEALTHY" + }, + { + "deploymentName": "default_MangoStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_OrangeStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_PearStand", + "status": "HEALTHY" + } + ] + } + ] + }, + "createdAt": "2023-09-18T07:55:23Z", + "deleteAt": "1969-12-31T23:59:59Z" + } + ] + } + ``` + #### List all services in all namespaces -``` +```text GET {{baseUrl}}/apis/v1alpha2/services ``` +Examples: + +* Request + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/services' \ + -H 'accept: application/json' + ``` + +* Response: + + ```json + { + "services": [ + { + "name": "test-v1", + "namespace": "ray-system", + "user": "user", + "serveDeploymentGraphSpec": { + "importPath": "fruit.deployment_graph", + "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"\n", + "serveConfigs": [ + { + "deploymentName": "OrangeStand", + "replicas": 1, + "userConfig": "price: 2", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "PearStand", + "replicas": 1, + "userConfig": "price: 1", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "FruitMarket", + "replicas": 1, + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "DAGDriver", + "replicas": 1, + "routePrefix": "/", + "actorOptions": { + "cpusPerActor": 0.1 + } + } + ] + }, + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "replicas": 1, + "minReplicas": 5, + "maxReplicas": 1, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "rayServiceStatus": { + "rayServiceEvents": [ + { + "id": "test-v1.1785ef1932a53822", + "name": "test-v1-test-v1.1785ef1932a53822", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "ServiceUnhealthy", + "message": "The service is in an unhealthy state. Controller will perform a round of actions in 10s.", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1962bfb4c2", + "name": "test-v1-test-v1.1785ef1962bfb4c2", + "createdAt": "2023-09-18T07:55:27Z", + "firstTimestamp": "2023-09-18T07:55:27Z", + "lastTimestamp": "2023-09-18T07:55:30Z", + "reason": "WaitForServeDeploymentReady", + "message": "Put \"http://test-v1-raycluster-vcvz2-head-svc.ray-system.svc.cluster.local:52365/api/serve/deployments/\": dial tcp 10.96.98.134:52365: connect: connection refused", + "type": "Normal", + "count": 3 + }, + { + "id": "test-v1.1785ef1b0f379aa9", + "name": "test-v1-test-v1.1785ef1b0f379aa9", + "createdAt": "2023-09-18T07:55:34Z", + "firstTimestamp": "2023-09-18T07:55:34Z", + "lastTimestamp": "2023-09-18T07:55:34Z", + "reason": "SubmittedServeDeployment", + "message": "Controller sent API request to update Serve deployments on cluster test-v1-raycluster-vcvz2", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1bbaa5aa5f", + "name": "test-v1-test-v1.1785ef1bbaa5aa5f", + "createdAt": "2023-09-18T07:55:37Z", + "firstTimestamp": "2023-09-18T07:55:37Z", + "lastTimestamp": "2023-09-18T07:55:39Z", + "reason": "ServiceNotReady", + "message": "The service is not ready yet. Controller will perform a round of actions in 2s.", + "type": "Normal", + "count": 2 + }, + { + "id": "test-v1.1785ef1cd4c74326", + "name": "test-v1-test-v1.1785ef1cd4c74326", + "createdAt": "2023-09-18T07:55:41Z", + "firstTimestamp": "2023-09-18T07:55:41Z", + "lastTimestamp": "2023-09-18T08:05:27Z", + "reason": "Running", + "message": "The Serve applicaton is now running and healthy.", + "type": "Normal", + "count": 277 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1920827324", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1920827324", + "createdAt": "2023-09-18T07:55:25Z", + "firstTimestamp": "2023-09-18T07:55:25Z", + "lastTimestamp": "2023-09-18T07:55:25Z", + "reason": "Created", + "message": "Created service test-v1-raycluster-vcvz2-head-svc", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1926720c3b", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1926720c3b", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created head pod test-v1-raycluster-vcvz2-head-959rs", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 1 + } + ], + "rayClusterName": "test-v1-raycluster-vcvz2", + "serveApplicationStatus": [ + { + "name": "default", + "status": "RUNNING", + "serveDeploymentStatus": [ + { + "deploymentName": "default_FruitMarket", + "status": "HEALTHY" + }, + { + "deploymentName": "default_MangoStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_OrangeStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_PearStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_DAGDriver", + "status": "HEALTHY" + } + ] + } + ] + }, + "createdAt": "2023-09-18T07:55:23Z", + "deleteAt": "1969-12-31T23:59:59Z" + } + ] + } + ``` + #### Get service by its name and namespace -``` +```text GET {{baseUrl}}/apis/v1alpha2/namespaces//services/ ``` +Examples: + +* Request: + + ```sh + curl --silent -X 'GET' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/services/test-v1' \ + -H 'accept: application/json' + ``` + +* Response: + + ```json + { + "name": "test-v1", + "namespace": "ray-system", + "user": "user", + "serveDeploymentGraphSpec": { + "importPath": "fruit.deployment_graph", + "runtimeEnv": "working_dir: \"https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip\"\n", + "serveConfigs": [ + { + "deploymentName": "OrangeStand", + "replicas": 1, + "userConfig": "price: 2", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "PearStand", + "replicas": 1, + "userConfig": "price: 1", + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "FruitMarket", + "replicas": 1, + "actorOptions": { + "cpusPerActor": 0.1 + } + }, + { + "deploymentName": "DAGDriver", + "replicas": 1, + "routePrefix": "/", + "actorOptions": { + "cpusPerActor": 0.1 + } + } + ] + }, + "serviceUnhealthySecondThreshold": 900, + "deploymentUnhealthySecondThreshold": 300, + "clusterSpec": { + "headGroupSpec": { + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "serviceType": "NodePort", + "rayStartParams": { + "dashboard-host": "0.0.0.0", + "metrics-export-port": "8080" + } + }, + "workerGroupSpec": [ + { + "groupName": "small-wg", + "computeTemplate": "default-template", + "image": "rayproject/ray:2.6.3-py310-aarch64", + "replicas": 1, + "minReplicas": 5, + "maxReplicas": 1, + "rayStartParams": { + "node-ip-address": "$MY_POD_IP" + } + } + ] + }, + "rayServiceStatus": { + "rayServiceEvents": [ + { + "id": "test-v1.1785ef1932a53822", + "name": "test-v1-test-v1.1785ef1932a53822", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "ServiceUnhealthy", + "message": "The service is in an unhealthy state. Controller will perform a round of actions in 10s.", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1962bfb4c2", + "name": "test-v1-test-v1.1785ef1962bfb4c2", + "createdAt": "2023-09-18T07:55:27Z", + "firstTimestamp": "2023-09-18T07:55:27Z", + "lastTimestamp": "2023-09-18T07:55:30Z", + "reason": "WaitForServeDeploymentReady", + "message": "Put \"http://test-v1-raycluster-vcvz2-head-svc.ray-system.svc.cluster.local:52365/api/serve/deployments/\": dial tcp 10.96.98.134:52365: connect: connection refused", + "type": "Normal", + "count": 3 + }, + { + "id": "test-v1.1785ef1b0f379aa9", + "name": "test-v1-test-v1.1785ef1b0f379aa9", + "createdAt": "2023-09-18T07:55:34Z", + "firstTimestamp": "2023-09-18T07:55:34Z", + "lastTimestamp": "2023-09-18T07:55:34Z", + "reason": "SubmittedServeDeployment", + "message": "Controller sent API request to update Serve deployments on cluster test-v1-raycluster-vcvz2", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1.1785ef1bbaa5aa5f", + "name": "test-v1-test-v1.1785ef1bbaa5aa5f", + "createdAt": "2023-09-18T07:55:37Z", + "firstTimestamp": "2023-09-18T07:55:37Z", + "lastTimestamp": "2023-09-18T07:55:39Z", + "reason": "ServiceNotReady", + "message": "The service is not ready yet. Controller will perform a round of actions in 2s.", + "type": "Normal", + "count": 2 + }, + { + "id": "test-v1.1785ef1cd4c74326", + "name": "test-v1-test-v1.1785ef1cd4c74326", + "createdAt": "2023-09-18T07:55:41Z", + "firstTimestamp": "2023-09-18T07:55:41Z", + "lastTimestamp": "2023-09-18T08:36:25Z", + "reason": "Running", + "message": "The Serve applicaton is now running and healthy.", + "type": "Normal", + "count": 837 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1920827324", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1920827324", + "createdAt": "2023-09-18T07:55:25Z", + "firstTimestamp": "2023-09-18T07:55:25Z", + "lastTimestamp": "2023-09-18T07:55:25Z", + "reason": "Created", + "message": "Created service test-v1-raycluster-vcvz2-head-svc", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef1926720c3b", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef1926720c3b", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created head pod test-v1-raycluster-vcvz2-head-959rs", + "type": "Normal", + "count": 1 + }, + { + "id": "test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "name": "test-v1-test-v1-raycluster-vcvz2.1785ef192cb1eb9d", + "createdAt": "2023-09-18T07:55:26Z", + "firstTimestamp": "2023-09-18T07:55:26Z", + "lastTimestamp": "2023-09-18T07:55:26Z", + "reason": "Created", + "message": "Created worker pod ", + "type": "Normal", + "count": 1 + } + ], + "rayClusterName": "test-v1-raycluster-vcvz2", + "serveApplicationStatus": [ + { + "name": "default", + "status": "RUNNING", + "serveDeploymentStatus": [ + { + "deploymentName": "default_DAGDriver", + "status": "HEALTHY" + }, + { + "deploymentName": "default_FruitMarket", + "status": "HEALTHY" + }, + { + "deploymentName": "default_MangoStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_OrangeStand", + "status": "HEALTHY" + }, + { + "deploymentName": "default_PearStand", + "status": "HEALTHY" + } + ] + } + ] + }, + "createdAt": "2023-09-18T07:55:23Z", + "deleteAt": "1969-12-31T23:59:59Z" + } + ``` #### Delete service by its name and namespace -``` +```text DELETE {{baseUrl}}/apis/v1alpha2/namespaces//services/ ``` -## Swagger Support +Examples: -1. Download Swagger UI from [Swagger-UI](https://swagger.io/tools/swagger-ui/download/). In this case, we use `swagger-ui-3.51.2.tar.gz` -2. Unzip package and copy `dist` folder to `third_party` folder -3. Use `go-bindata` to generate go code from static files. +* Request -``` -mkdir third_party -tar -zvxf ~/Downloads/swagger-ui-3.51.2.tar.gz /tmp -mv /tmp/swagger-ui-3.51.2/dist third_party/swagger-ui + ```sh + curl --silent -X 'DELETE' \ + 'http://localhost:31888/apis/v1alpha2/namespaces/ray-system/services/test-v1' \ + -H 'accept: application/json' + ``` -cd apiserver/ -go-bindata --nocompress --pkg swagger -o pkg/swagger/datafile.go ./third_party/swagger-ui/... -``` +* Response + + ```json + {} + ```