[serve] Investigate if serve/fastapi works with pydantic>=2.0

[edoakes]

Pydantic 2.0 broke compatibility with Ray (see #37019), so we currently have pydantic<2.0 pinned in our dependencies.

There is a fix for Ray core that will get cherry-picked to Ray 2.6: #37055, but there may be an issue between FastAPI and pydantic that would prevent us from removing the pin.

Someone from the Serve team needs to investigate if Serve/FastAPI will work with pydantic>=2.0 after the above fix is merged.

The first step is likely running the CI tests with pydantic pinned to the most recent version.

The outcome of this is a decision between:

1. Removing the pin for Ray 2.6 because we're confident that pydantic>=2.0 works with both core and serve.
2. Leaving the pin in for Ray 2.6 because fastapi/serve do not yet work with pydantic>=2.0.

(2) is the more conservative option which we should go with by default if we are unsure.

[edoakes]

@alanwguo and @rkooo567 have the most context on the Pydantic issue

[edoakes]

Basic serve tests are failing due to pydantic API changes:

schema.py:242: in DeploymentSchema
    def num_replicas_and_autoscaling_config_mutually_exclusive(cls, values):
/Users/eoakes/miniforge3/envs/ray/lib/python3.8/site-packages/pydantic/deprecated/class_validators.py:222: in root_validator
    return root_validator()(*__args)  # type: ignore
/Users/eoakes/miniforge3/envs/ray/lib/python3.8/site-packages/pydantic/deprecated/class_validators.py:228: in root_validator
    raise PydanticUserError(
E   pydantic.errors.PydanticUserError: If you use `@root_validator` with pre=False (the default) you MUST specify `skip_on_failure=True`. Note that `@root_validator` is deprecated and should be replaced with `@model_validator`.
E
E   For further information visit https://errors.pydantic.dev/2.0.2/u/root-validator-pre-skip

Updating that to add the suggested flag, I then get:

ImportError while loading conftest '/Users/eoakes/code/ray/python/ray/serve/tests/conftest.py'.
__init__.py:4: in <module>
    from ray.serve.api import (
api.py:15: in <module>
    from ray.serve.built_application import BuiltApplication
built_application.py:7: in <module>
    from ray.serve.deployment import Deployment
deployment.py:14: in <module>
    from ray.serve.context import get_global_client
context.py:12: in <module>
    from ray.serve._private.client import ServeControllerClient
_private/client.py:25: in <module>
    from ray.serve._private.deploy_utils import get_deploy_args
_private/deploy_utils.py:8: in <module>
    from ray.serve.schema import ServeApplicationSchema
schema.py:134: in <module>
    class DeploymentSchema(
/Users/eoakes/miniforge3/envs/ray/lib/python3.8/site-packages/pydantic/_internal/_model_construction.py:124: in __new__
    cls: type[BaseModel] = super().__new__(mcs, cls_name, bases, namespace, **kwargs)  # type: ignore
/Users/eoakes/miniforge3/envs/ray/lib/python3.8/abc.py:85: in __new__
    cls = super().__new__(mcls, name, bases, namespace, **kwargs)
E   TypeError: __init_subclass__() takes no keyword arguments

[sihanwang41]

Hi @edoakes & @shrekris-anyscale , how is the traceback related to the fastapi? (I am seeing @shrekris-anyscale add version control to fastapi)

[sihanwang41]

Hi @edoakes & @shrekris-anyscale , how is the traceback related to the fastapi? (I am seeing @shrekris-anyscale add version control to fastapi)

sync offline with @shrekris-anyscale, trying to not add any version control for fastapi.

[ddelange]

ref #40451

[shrekris-anyscale]

We've merged the changes to make Ray compatible with Pydantic 2.5+. You can start using Pydantic 2.5+ with Ray 2.9, which should be out at the end of December.

These changes should also be in the Ray nightly, so feel free to try them out!