[serve] eagerly create router for handles held by proxy

[zcin]

Why are these changes needed?

Eagerly create the router for handles held by the proxy actor, so that the controller can send the replica sets + their actor handles for each deployment upon proxy startup, instead of waiting for the first request to come in. This helps with improving controller fault tolerance.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[GeneDer]

Would be nice to add a new test maybe try to update routes on start and see the differences with/ without this change. But code itself LGTM

[edoakes]

nit: would prefer that we add an argument to get_handle instead of adding a private method call here

[zcin]

sounds good, added _lazily_initialize_router to get_deployment_handle and DeploymentHandle constructor. This is set to false in the proxy.

[zcin]

@edoakes Removed the parameter for now because it causes the router to be initialized before options passed in .options() are picked up by the initialized router, and there is no super clean way of adding the param and fixing that. Reverted to manually calling private method, will think more about how to add an official parameter for this.

[zcin]

Would be nice to add a new test maybe try to update routes on start and see the differences with/ without this change. But code itself LGTM

I've added tests for the new parameter I added for get handle. As for testing this in the proxy, I'll be adding more comprehensive e2e tests once more changes are made in later PRs!