Deprecate built in ingress

[kevin85421]

Why are these changes needed?

As we mentioned in ingress.md, we do not recommend using built-in ingress support in a production environment with complex routing requirements.

KubeRay built-in ingress support: KubeRay will help users create a Kubernetes ingress when spec.headGroupSpec.enableIngress: true. Currently, the built-in support only supports simple NGINX setups. Note that users still need to install ingress controller by themselves. We do not recommend using built-in ingress support in a production environment with complex routing requirements.

• (+) Based on [Feature] Can ray be deployed and run on k8s version 1.18 #905 (comment), the ingress support also causes some limitations on KubeRay/Kubernetes compatibility.
• (-) Break the backward compatibility, but it is easy to fix it by setting up an ingress manually.

Related issue number

#905

With the PR, the KubeRay operator will not report the error in #905 (comment). However, the RayCluster chart and YAML are still incompatible with Kubernetes 1.18. Will work on the following if this PR is approved by the community.

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • This PR is not tested :(

# Step 1: Create a RayCluster.
kind create cluster --image=kindest/node:v1.18.15

# Step 2: Clone this branch, and build the Docker image.
# Step 3: Load the Docker image into the Kind cluster.
kind load docker-image controller:latest

# Step 4: Install the KubeRay operator chart with the new image
helm install kuberay-operator kuberay/kuberay-operator --version 0.4.0 --set image.repository=controller,image.tag=latest

[Jeffwan]

This is a breaking change and I don't suggest to deprecate it immediately. As you said, you don't suggest to use with complex rules. However, lots of user use simplified rules. Why not to make it optional but deleting the code? Even if we want to deprecate a feature, it should have an exit plan.

[kevin85421]

This is a breaking change and I don't suggest to deprecate it immediately. As you said, you don't suggest to use with complex rules. However, lots of user use simplified rules. Why not to make it optional but deleting the code? Even if we want to deprecate a feature, it should have an exit plan.

Thank @Jeffwan for the comments!

I agree with you that the deprecation is a bit aggressive. This is a tradeoff between:

• Pros:

  • Should KubeRay support Kubernetes 1.18? If yes, we need to deprecate the built-in ingress support.
  • Performance improvement (not sure, but the reconciler does not need to reconcile ingress events after it is deprecated.)

• Cons:

  • The deprecation breaks the backward compatibility. However, the cost is not that expensive because the feature starts to work in release 0.4.0 (#646), and we have a document to deemphasize the built-in ingress support at that moment.

A user wants to deploy KubeRay on Kubernetes 1.18 (#905). If the KubeRay community does not want to support Kubernetes 1.18, it it unnecessary to deprecate the built-in ingress support. On the other hand, we need to deprecate it. Maybe we can discuss this topic in the next community sync.

[Jeffwan]

@kevin85421 if supporting 1.18 is the case, I still suggest not to delete it. As you know, ingress API was changed as well and there're breaking changes for sure. Another thing to notice is 1.18 was sunset for long time in most public clouds. I suggest we open a new branch and merge the code there, WDYT? Users can build images by themselves.