Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error message in the body of BrokeredMessage #424

Closed
ikoblik opened this issue Apr 30, 2014 · 8 comments
Closed

Error message in the body of BrokeredMessage #424

ikoblik opened this issue Apr 30, 2014 · 8 comments

Comments

@ikoblik
Copy link

ikoblik commented Apr 30, 2014

Context

We're using Azure SKD version 0.4.6 to read from and post messages to service bus queues. We always use PeekLock read strategy. In terms of volume we process somewhere around 20 messages per second or less.

What happens

Rather infrequently we receive BrokeredMessages that contain a Service Bus error string in the body of the message (i.e. BrokeredMessage.gotBody()). So far we saw 2 different errors in the message body:

<Error><Code>500</Code><Detail>The server was unable to process the request; please retry the operation. 
If the problem persists, please contact your Service Bus administrator and provide the tracking id..
TrackingId:00012cd4-255d-46ea-9cf2-9d37019d76d1_G26,TimeStamp:4/24/2014 10:23:32 AM</Detail></Error>

and

<Error><Code>500</Code><Detail>Error during communication with Service Bus. 
Check the connection information, then retry..
TrackingId:D369ac70-9be4-4f9a-b7bb-e1dfc73a5f8f_G28,TimeStamp:4/24/2014 11:17:02 AM</Detail></Error>

We detect these errors because we pass messages in the Protobuffer format, but it might be an issue for some clients who pass raw text data.

What we expect

We expect from the SDK to raise a ServiceException. At a first glance, it seems that the SDK fails to detect errors coming form ServiceBus and passes them as a normal respons to its clients.

@ikoblik ikoblik changed the title Error message in the body BrokeredMessage Error message in the body of BrokeredMessage Apr 30, 2014
@ikoblik
Copy link
Author

ikoblik commented May 1, 2014

Added library version and some additional context to help with the investigation. Please let me know if you need additional information, I'd be happy to provide it.

@ikoblik
Copy link
Author

ikoblik commented May 7, 2014

Here's another error we received recently in the body of the message:

<Error><Code>403</Code><Detail>Messaging entity \'<<CUT>>:Queue:<<CUT>>\' is currently disabled..TrackingId:0a06cff4-1ef4-4974-a092-46d20c8989f8_G15,TimeStamp:5/7/2014 11:59:22 AM</Detail></Error>"

@bradygaster-zz
Copy link

@ikoblik thanks for the bug report. We'll sync with our partners in Service Bus to see what can be done about this. We just released the 0.6.0 version of the Java SDK - could you check to see if this defect is still there?

@bcotrim
Copy link

bcotrim commented Aug 1, 2014

Hello,

Where can I find the 0.6.0 version os the Java SDK?

@bradygaster-zz
Copy link

@bcotrim you can find it in Maven here: http://search.maven.org/#browse%7C1671162511

@jianghaolu
Copy link
Contributor

@ikoblik We are closing this issue assuming the bug is already fixed in the new release. Please feel free to let us know if it still repros.

@venkat22
Copy link

This issue got reintroduced back in 0.7 version of the SDK. Please reopen this issue again until it get resolved again.

@jianghaolu jianghaolu reopened this Jun 12, 2015
@martinsawicki
Copy link

the service bus is not in this repo anymore, so if this is still an issue, it should go there:
https://github.com/azure/azure-service-bus-java

mssfang pushed a commit to mssfang/azure-sdk-for-java that referenced this issue Feb 6, 2019
conniey added a commit that referenced this issue Apr 30, 2019
* Update Apache Proton-J dependency (0.29.0 --> 0.31.0) (#407)

* PartitionReceiver - add a method that provides an EventPosition which corresponds to an EventData returned last by the receiver (#408)

* Support IsPartitionEmpty property for PartitionRuntimeInformation (#399)

* Move setPrefetchCount API to the ReceiverOptions class from the PartitionReceiver and update the settings of Default & Max Prefetch count (#410)

This pull request includes two major changes related to Prefetch API.

1) Move setPrefetchCount API to the ReceiverOptions class so that prefetch value specified by a user can be used instead of using default value when communicating to the service during link open and initializing a receiver. This change also addresses the receiver stuck issue caused by setPrefetchAPI in a race condition.

2) Change the default value and set the upper bound of the prefetch count. Note that prefetch count should be greater than or equal to maxEventCount which can be set when either a) calling receive() API or b) implementing the getMaxEventCount API of the SessionReceiverHandler interface.

* Fixes several issues in the reactor related components (#411)

This pull request contains the following changes.

1) Finish pending tasks when recreating the reactor and make sure pending calls scheduled on the old reactor get complete.
2) Fix the session open timeout issue which can result in NPE in proton-J engine.
3) Make session open timeout configurable and use the value of OperationTimeout.
4) Update the message of exceptions and include an entity name in the exception message.
5) API change - use ScheduledExecutorService.
6) Improve tracing.

* Implement comparable on EventData (#395)

* Update receive/send link creation logic and improve tracing (#414)

* Prep for releasing client 2.0.0 and EPH 2.2.0 (#415)

* Ensure that links are closed when transport error occurrs (#417)

* ensure links are recreated on transport/connection failure
* update API document for EventProcessorOptions class
* add traces for link create/close case

* Prep for releasing client 2.1.0 and EPH 2.3.0 (#418)

* Update prefetch sendflow logic and increment version for new release (#420)

* Fix args for proxy auth call to Authenticator (#421)

* Prepare EPH 2.3.4 release (#423)

* Prepare EPH 2.4.0 release (#423) (#424)

* Handle proton:io errors with meaningful error msg (#427)

* Handle proton:io errors with meaningful error msg

* Use Proton-supplied message if present

* Minor changes to lease scanner (#428)

* Add logging if the scanner threw an exception.
* Change logging level to warn when scanner shuts down for any reason.
* Scanner can call EventProcessorOptions.notifyOfException, which calls user code. Change notifyOfException to defensively catch any exceptions coming out of user code.

* Make EventData.SystemProperties completely public (#435)

Porting testability changes from .NET Core to Java: provide full access to EventData's SystemProperties so that a complete EventData can be fabricated in tests.

* Digest Support: init first connection with null headers (#431)

Related to Azure/qpid-proton-j-extensions#10

* Fix lease scanner issues when Storage unreachable (#434)

This fix is for issue #432. There are two parts:

AzureStorageCheckpointLeaseManager performs certain Storage actions within a forEach. If those actions fail, the StorageException gets wrapped in a NoSuchElementException. Catch those and strip off the NoSuchElementException, then handle the StorageException in the existing way.

The unexpected NoSuchElementExceptions were not being caught anywhere and the scanner thread was dying without rescheduling itself. Added code in PartitionMananger.scan to catch any exceptions that leak out of PartitionScanner and reschedule the scanner unless the host instance is shutting down.

* message receiver - fix null pointer error and ensure that receive link is recreated upon a failure (#439)

* message receiver/sender - fix null pointer error and ensure that receive/send link is recreated on a failure.

* Update version numbers for release (#440)

* Update prefetch count for a receiver (#441)

* Fix an issue of creating multiple sessions for $management & $cbs channel for a single connection and improve logging (#443)

* Fix an issue of creating multiple sessions for $management & $cbs for a connection and improve logging

* Running through java files and double checking changes

* Fix casing on test names

* Ignore testcases that hang.

* Fix NullPointerException when there is no inner exception

* Move parent node to the top of the file.

* Update version numbers in spotbugs-reporting

* Increasing wait time until event hub scheduler is completed.
conniey added a commit that referenced this issue May 3, 2019
* Update Apache Proton-J dependency (0.29.0 --> 0.31.0) (#407)

* PartitionReceiver - add a method that provides an EventPosition which corresponds to an EventData returned last by the receiver (#408)

* Support IsPartitionEmpty property for PartitionRuntimeInformation (#399)

* Move setPrefetchCount API to the ReceiverOptions class from the PartitionReceiver and update the settings of Default & Max Prefetch count (#410)

This pull request includes two major changes related to Prefetch API.

1) Move setPrefetchCount API to the ReceiverOptions class so that prefetch value specified by a user can be used instead of using default value when communicating to the service during link open and initializing a receiver. This change also addresses the receiver stuck issue caused by setPrefetchAPI in a race condition.

2) Change the default value and set the upper bound of the prefetch count. Note that prefetch count should be greater than or equal to maxEventCount which can be set when either a) calling receive() API or b) implementing the getMaxEventCount API of the SessionReceiverHandler interface.

* Fixes several issues in the reactor related components (#411)

This pull request contains the following changes.

1) Finish pending tasks when recreating the reactor and make sure pending calls scheduled on the old reactor get complete.
2) Fix the session open timeout issue which can result in NPE in proton-J engine.
3) Make session open timeout configurable and use the value of OperationTimeout.
4) Update the message of exceptions and include an entity name in the exception message.
5) API change - use ScheduledExecutorService.
6) Improve tracing.

* Implement comparable on EventData (#395)

* Update receive/send link creation logic and improve tracing (#414)

* Prep for releasing client 2.0.0 and EPH 2.2.0 (#415)

* Ensure that links are closed when transport error occurrs (#417)

* ensure links are recreated on transport/connection failure
* update API document for EventProcessorOptions class
* add traces for link create/close case

* Prep for releasing client 2.1.0 and EPH 2.3.0 (#418)

* Update prefetch sendflow logic and increment version for new release (#420)

* Fix args for proxy auth call to Authenticator (#421)

* Prepare EPH 2.3.4 release (#423)

* Prepare EPH 2.4.0 release (#423) (#424)

* Handle proton:io errors with meaningful error msg (#427)

* Handle proton:io errors with meaningful error msg

* Use Proton-supplied message if present

* Minor changes to lease scanner (#428)

* Add logging if the scanner threw an exception.
* Change logging level to warn when scanner shuts down for any reason.
* Scanner can call EventProcessorOptions.notifyOfException, which calls user code. Change notifyOfException to defensively catch any exceptions coming out of user code.

* Make EventData.SystemProperties completely public (#435)

Porting testability changes from .NET Core to Java: provide full access to EventData's SystemProperties so that a complete EventData can be fabricated in tests.

* Digest Support: init first connection with null headers (#431)

Related to Azure/qpid-proton-j-extensions#10

* Fix lease scanner issues when Storage unreachable (#434)

This fix is for issue #432. There are two parts:

AzureStorageCheckpointLeaseManager performs certain Storage actions within a forEach. If those actions fail, the StorageException gets wrapped in a NoSuchElementException. Catch those and strip off the NoSuchElementException, then handle the StorageException in the existing way.

The unexpected NoSuchElementExceptions were not being caught anywhere and the scanner thread was dying without rescheduling itself. Added code in PartitionMananger.scan to catch any exceptions that leak out of PartitionScanner and reschedule the scanner unless the host instance is shutting down.

* message receiver - fix null pointer error and ensure that receive link is recreated upon a failure (#439)

* message receiver/sender - fix null pointer error and ensure that receive/send link is recreated on a failure.

* Update version numbers for release (#440)

* Update prefetch count for a receiver (#441)

* Fix an issue of creating multiple sessions for $management & $cbs channel for a single connection and improve logging (#443)

* Fix an issue of creating multiple sessions for $management & $cbs for a connection and improve logging

* Update version numbers for new release (#444)

* Update spotbugs.xml report versions
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants