[serve] Change Gradio integration to take a builder function to avoid serialization issues

[zcin]

Signed-off-by: Cindy Zhang [email protected]

Why are these changes needed?

New versions of Gradio make Gradio objects un-serializable in Ray. This makes it impossible to directly pass in a Gradio object to GradioIngress, so to avoid this the API is changed to accept a builder function that constructs the Gradio object on the Ray cluster.

Related issue number

Closes #31161

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[sihanwang41]

The fix looks good to me. General question: does it mean we will not support Gradio 3.5 in this case?

[zcin]

The fix looks good to me. General question: does it mean we will not support Gradio 3.5 in this case?

No, earlier versions should still be supported. We are just sidestepping the serialization issue by changing the API of our integration - functionality is still the same, the Gradio object is just built remotely on the cluster instead of locally to avoid serialization.

[zcin]

Test linux://rllib:test_memory_leak_a3c is flaky on master, seems unrelated.

[sihanwang41]

The fix looks good to me. General question: does it mean we will not support Gradio 3.5 in this case?

No, earlier versions should still be supported. We are just sidestepping the serialization issue by changing the API of our integration - functionality is still the same, the Gradio object is just built remotely on the cluster instead of locally to avoid serialization.

[sihanwang41]

sorry, click wrong button....