Commits on Jan 30, 2024

1. [improve][broker] Do not close the socket if lookup failed due to Loc… …

   …kBusyException
   
   ### Motivation
   
   When a broker restarted, there is a case in
   `NamespaceService#findBrokerServiceUrl`:
   1. `ownershipCache.getOwnerAsync(bundle)` got an empty data, then
      `searchForCandidateBroker` will be called
   2. The broker itself was elected as the candidate broker.
   3. Meanwhile, the other broker has acquired the distributed lock of the
      bundle, then `ownershipCache.tryAcquiringOwnership` will fail with
   
   ```java
   lookupFuture.completeExceptionally(new PulsarServerException(
           "Failed to acquire ownership for namespace bundle " + bundle, exception));
   ```
   
   See apache/pulsar-client-cpp#390 for the real
   world case.
   
   Then in `TopicLookupBase#handleLookupError`, this exception will be
   wrapped into a `ServiceNotReady` error to client.
   
   This case happens very frequently in our production environment when a
   broker restarted. If there is a `PulsarClient` that has many producers
   or consumers, the connection will be closed, which results in many
   reconnections, which brings much pressure to the cluster.
   
   ### Modifications
   
   In `handleLookupError`, check the `PulsarServerException` and unwrap the
   `CompletionException`. If the unwrapped exception is
   `MetadataStoreException`, return the `MetadataError` to avoid closing
   the connection at client side.
   
   Add `testLookupConnectionNotCloseIfFailedToAcquireOwnershipOfBundle` to
   simulate the case and verify the socket won't be closed.

   

   BewareMyPower committed Jan 30, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for c4bd42f
   • Browse repository at this point

   Copy the full SHA

   c4bd42f View commit details

   Browse the repository at this point in the history