[Serve] [KubeRay] Add flag that allows serve build to print Kubernetes-formatted output

[shrekris-anyscale]

Why are these changes needed?

serve build lets users autogenerate a config file for their Serve deployment graph. Serve natively requires the config file to use camel casing for its keys. However, the KubeRay config requires camel-cased keys. This makes it difficult for users to autogenerate a config for Kubernetes.

This change adds the --kuberenets-format/-k option, which generates a Serve config with camel-cased keys, so it can be more easily copy-pasted into the Kubernetes config.

Related issue number

Closes #28831.

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
    • This change adds unit tests for helper functions and serve build.

[shrekris-anyscale]

I think this warning can be worded more concisely and clearly. Does anyone have suggestions?

[simon-mo]

Can we just json serialize it for the user instead?

[sihanwang41]

+1, it would be great to remove this manual action from the customer side.

[shrekris-anyscale]

We could json serialize it. The output would look like this:

importPath: modded_fruit:deployment_graph

runtimeEnv: '{}'

host: 0.0.0.0

port: 8000

deployments:

- name: MangoStand
  userConfig: '{"price": 3, "fake_prop": "what_a_str!", "5": 2, "22": "hello"}'
  rayActorOptions:
    numCpus: 0.1

- name: OrangeStand
  userConfig: '{"price": 2, "fake_prop": "what_a_str!", "5": 2, "22": "hello"}'
  rayActorOptions:
    numCpus: 0.1

- name: PearStand
  userConfig: '{"price": 4, "fake_prop": "what_a_str!", "5": 2, "22": "hello"}'
  rayActorOptions:
    numCpus: 0.1

- name: FruitMarket
  numReplicas: 2
  rayActorOptions:
    numCpus: 0.1

- name: DAGDriver
  routePrefix: /

This is a bit more convoluted than e.g. this YAML spec where each key in the spec is listed on a newline with a pipe (|) character:

- name: OrangeStand
        numReplicas: 1
        userConfig: |
          price: 2
        rayActorOptions:
          numCpus: 0.1

Note: I anticipated that the JSON-serialized version would look worse than it is. Now that I've tried it, I actually don't think it looks too bad. I'd prefer to JSON-serialize and remove the manual step. What do you think @simon-mo @sihanwang41?

[sihanwang41]

The output looks good to me. Without manual step, more convenient and less error-prone

[shrekris-anyscale]

I've updated the code to JSON-serialize the fields that Kubernetes config expects to be strings:

• runtime_env
• deployment > user_config
• deployment > ray_actor_options > runtime_env
• deployment > ray_actor_options > resources

[simon-mo]

Suggested change

	words = snake_str.lstrip("_").strip("_").split("_")
	words = snake_str.strip("_").split("_")

strip does lstrip's job already

[shrekris-anyscale]

Thanks for letting me know. I removed the lstrip call.

[simon-mo]

this isn't recursive right?

[shrekris-anyscale]

No, it's not recursive.

[simon-mo]

Can we just json serialize it for the user instead?

[simon-mo]

Please add a note into the documentation as well.

[simon-mo]

quick non-blocking question here..
why do we have kubernetes_dict calls dict instead of calling dict first, then have a pure function that transform the config to k8s format? (basically why a method while the stuff is pretty pure functional)

[shrekris-anyscale]

Good question– the Kubernetes dictionary format had a few rules that were schema-specific:

• All the keys in the config (including deployment dictionaries and ray_actor_options) should be camel-cased.
• Some of the dictionaries need to be json-serialized:
  • runtime_env
  • deployment_dictionary > user_config
  • deployment dictionary > ray_actor_options > runtime_env
  • deployment_dictionary > ray_actor_options > resources)

I made it a method because the second requirement made it feel too specific to be a pure function.