[Serve] deploy_group method deletes all deployments

[mfojtak]

What happened + What you expected to happen

serve.run calls deploy_group which wipes out all previosly deployed deployments.

This is because:

deployment_names_to_delete = all_deployments_names.difference(
                new_deployments_names
            )

in client.py

this finds difference between sets - which is all existing deployments.
It should be using intersection - which finds new deployments if they already exist

Also there is hardcoded value for remove_past_deployments in serve.run:

client.deploy_group(
        parameter_group, _blocking=_blocking, remove_past_deployments=True
    )

Thsi should be parametric. Btw. setting this to False fixes the problem and serve.run works as expected - leaving existing deployments intact and updating new ones.

Versions / Dependencies

2.0.0

Reproduction script

serve.run(deployment_A)

serve.run(deployment_B) #deployment_A will be deleted

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[sihanwang41]

This is supported with new api: https://docs.ray.io/en/latest/serve/api/doc/ray.serve.run.html

you can specify name to each application deploy to avoid deployment deletion

[JoshKarpel]

@sihanwang41 is this option supported through the serve run CLI command and KubeRay's Serve interface?

[sihanwang41]

it is coming in the 2.4 for CLI.
Kuberay is not supported yet.

[JoshKarpel]

Thanks! Is there a ticket I can follow for support in KubeRay?

[sihanwang41]

ray-project/kuberay#985