feat: upgrade to new fastapi, update models to handle both pydantic v…

[timothyjlaurent]

…1 an v2, add matrix conditions for pydantic_v1 and pydantic_v2

What this PR does / why we need it:
This work builds on this PR #3273 that adds pydantic 2 support to the kserve package.

The following is copied from that PR:

• Follows Optional fields guidance for pydantic v2 ref
  *Overwrites protected_namespace in pydantic v2 to get rid of warning logs about namespace conflicts in pydantic ref. An example is shown as follows

E   UserWarning: Field "model_name" has conflict with protected namespace "model_".
E
E   You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.

In addition to handling those incompatibilities it also allows for both pydantic 1.x and 2.x versions.

To accomplish this:

• FastAPI is upgraded to the latest version. This version supports both pydantic 1.x and 2.x. As such, projects that add kserve as a dependency will also be able to use both pydantic versions.
• inspired by how https://github.com/docarray/docarray added support for v1 and v2 pydantic an is_pydantic_2 flag is used to change the underlying pydantic implementations
• each pydantic model that has incompatible syntax, ie nested Config class in retains the class if it is pydantic 1 and uses the model_config syntax if it is pydantic 2. Keys are changed where needed, eg schema_extra -> json_schema_extra.
• protected_namespace=() is set on the InferenceResponse model
• to test, again inspired by docarray's CI, a new matrix component pydantic-version is added to CI. As the default version will be pydantic 2, in the case that the version is pydantic_1 then the make target install_pydantic_1 is executed which downgrades pydantic to v1.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #3373

Type of changes
Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Feature/Issue validation/testing:

Please describe the tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

• Added matrix to test both pydantic 1 and pydantic 2

• Logs

Special notes for your reviewer:

1. Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Checklist:

• Have you added unit/e2e tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

Release note:

Upgrades FastAPI version and now supports Pydantic v1.x and v2.x. This allows dependent projects to upgrade their Pydantic version as needed.

[sivanantha321]

@timothyjlaurent Can you help rebase the PR and sign off the commits ?

[timothyjlaurent]

@sivanantha321 I've rebased and signed all the commits, can you take another look?

[sivanantha321]

@yuzisun Can you help trigger the tests ?

[timothyjlaurent]

We had a small bug in the last test and then I somehow borked my rebase -- I've cleaned up the commits on a fresh branch and think it should pass now.

[sivanantha321]

@timothyjlaurent I don't see a updated version of pydantic in poetry.lock file ? I think we can upgrade the pydantic version to >2.0.0 for model servers.

[timothyjlaurent]

@sivanantha321 Ah this is because we hack/python-release runs with --no-update and the sdk is compatible with pydantic v1 or v2.

I can either run it to update all packages, or I could add pydantic as an explicit dep to each model server. What do you think?

[sivanantha321]

@sivanantha321 Ah this is because we hack/python-release runs with --no-update and the sdk is compatible with pydantic v1 or v2.

I can either run it to update all packages, or I could add pydantic as an explicit dep to each model server. What do you think?

You can update the packages as long as it does not breaks anything

[timothyjlaurent]

@sivanantha321 I've updated the deps.

[yuzisun]

@timothyjlaurent Thanks for upgrading pydantic! Can you help take a look a look at the testing failures?
https://github.com/kserve/kserve/actions/runs/7993801929/job/21947239919?pr=3374

[timothyjlaurent]

@timothyjlaurent Thanks for upgrading pydantic! Can you help take a look a look at the testing failures? https://github.com/kserve/kserve/actions/runs/7993801929/job/21947239919?pr=3374

I took the time to figure out how to properly run the test suite locally -- needs to be done from the ./python directory.

I've updated the tests one notable workaround is that the RayServe test asserted b'{"name":"TestModel","ready":"True"}' <- notice the stringified bool. In pydantic V2 we correctly get an actual json bool for the ready param. I added a special case to the test and left comments explaining this behavior.

[sivanantha321]

I think it is a different bug altogether. Before moving to fastapi, it seems the health handler returned bool values,
but after the migration it started returning string values because how the type is defined ?
@yuzisun

[yuzisun]

The code seems to be returning bool, as @timothyjlaurent mentioned the issue is for the rayserve case?

[timothyjlaurent]

This is for the RayServe case.

In the previous assertion with pydantic v1 - we were getting {"name":"TestModel","ready":"True"} the string value, and testing for that. Seems like it was a bug that got "blessed" by enshrinining it in a testcase.
With the pyanticv2 we're (correctly) getting a bool value for the ready key (not a stringified python string) - This, I believe is the correct behavior we should expect.

[yuzisun]

@timothyjlaurent seems like it keeps failing after updating the poetry lock file
https://github.com/kserve/kserve/actions/runs/8087460553/job/22214167194?pr=3374

[yuzisun]

Tests passed, Thanks @timothyjlaurent for the great work!

/lgtm
/approve

[oss-prow-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: timothyjlaurent, yuzisun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [yuzisun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment