change ray version and enable auto scaling

[hansinikarunarathne]

Pull Request Template for Kubeflow manifests Issues

✏️ A brief description of the changes

Updated the ray version and enable autoScaling

🐛 If this PR is related to an issue, please put the link to the issue here.

#2742

✅ Contributor checklist

• Make sure you have tested with kustomize. See Installation Prerequisites
• All the commits have been signed-off (To pass the DCO check)
• Submit the Contributor License Agreements (To pass the cla/google check)

---

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

[juliusvonkohout]

There could be more outdated stuff that we should replace with upstream stuff.

For example this looks interesting. Maybe busybox is outdated.

        - name: init
          image: busybox:1.36
          # Change the cluster postfix if you don't have a default setting
          command: ['sh', '-c', "until nslookup $RAY_IP.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for K8s Service $RAY_IP; sleep 2; done"]

[juliusvonkohout]

I think

Ray seems to have documentation for istio support now ray-project/ray#44801
ray-project/kuberay#1005
ray-project/kuberay#2117

Istio is something for a second PR. lets do a thorough cleanup here first and merge

[juliusvonkohout]

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: juliusvonkohout

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• contrib/ray/OWNERS [juliusvonkohout]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hansinikarunarathne]

There could be more outdated stuff that we should replace with upstream stuff.

For example this looks interesting. Maybe busybox is outdated.

        - name: init
          image: busybox:1.36
          # Change the cluster postfix if you don't have a default setting
          command: ['sh', '-c', "until nslookup $RAY_IP.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for K8s Service $RAY_IP; sleep 2; done"]

can you elobrate on this comment more. It is little confusing

[hansinikarunarathne]

There could be more outdated stuff that we should replace with upstream stuff.

For example this looks interesting. Maybe busybox is outdated.

        - name: init
          image: busybox:1.36
          # Change the cluster postfix if you don't have a default setting
          command: ['sh', '-c', "until nslookup $RAY_IP.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for K8s Service $RAY_IP; sleep 2; done"]

Can we update the ray cluster yaml as this with using a headless service?
https://docs.ray.io/en/latest/cluster/kubernetes/k8s-ecosystem/istio.html#step-4-create-the-raycluster

I started to go through the documents but did not find any clue as to what you were saying.

[juliusvonkohout]

yes, just try something out :-)