[serve] proxy should ping replica immediately after receiving new actor handle #47036
Labels
enhancement
Request for new feature and/or capability
P1
Issue that should be fixed within a few weeks
serve
Ray Serve Related Issue
When proxies get an updated set of running replicas, it receives the actor handles for them from the controller. However the actor handle itself is only a pointer that contains the actor ID, but doesn't actually contain all actor info such as the actor address. This info is populated by the GCS upon first request made through that actor handle.
If GCS goes down before the proxy gets to make that first request, requests will hang in the duration of GCS of failure. To improve fault tolerance, the router should ping the replica immediately after receiving a new actor handle to populate the actor info.
The text was updated successfully, but these errors were encountered: