[BUG] Error processing list databricks workspace response.

[costichiulan]

Describe the bug
Different API response when listing Databricks workspaces between tenants. We are calling list databricks workspaces API in different tenants / different regions and response was changed in one specific tenant in the last days. Due to this difference azureresource-databricks library is throwing an error.

We are using sdk, but I also double checked with plain API call. We are using 2021-04-01-preview apiVersion.

In most tenants the following response is received, and it's processed correctly by sdk.
{ "value": [ { "properties": { ... } ... }, { "properties": { ... } ... } ] }

In another tenant the following response is received an sdk throws an error .

[ { "properties": { ... } ... }, { "properties": { ... } ... } ]

We are using com.azureresourcemanager-databricks-1.0.0-beta3
Due to this difference, SDK is throwing an error ( check stack trace )

Exception or Stack Trace
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type com.azure.resourcemanager.databricks.models.WorkspaceListResult from Array value (token JsonToken.START_ARRAY)
at [Source: (byte[])"[{"properties":{"managedResourceGroupId":"{resourceGroupId}","parameters":{"enableNoPublicIp":{"type":"Bool","value":false},"storageAccountName":{"type":"String","value":""},"storageAccountSkuName":{"type":"String","value":""},"vnetAddressPrefix":{"type":"String","value":"10.139"},"resourceTags":{"type":"Object","value":{"application":"databricks","databricks-envir"[truncated 3710 bytes]; line: 1, column: 1]
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
at reactor.core.publisher.Flux.blockLast(Flux.java:2761)
at com.azure.core.util.paging.ContinuablePagedByIteratorBase.requestPage(ContinuablePagedByIteratorBase.java:102)
at com.azure.core.util.paging.ContinuablePagedByItemIterable$ContinuablePagedByItemIterator.(ContinuablePagedByItemIterable.java:75)
at com.azure.core.util.paging.ContinuablePagedByItemIterable.iterator(ContinuablePagedByItemIterable.java:55)
at com.azure.core.util.paging.ContinuablePagedIterable.iterator(ContinuablePagedIterable.java:141)
at com.azure.resourcemanager.databricks.implementation.Utils$PagedIterableImpl.iterator(Utils.java:136

To Reproduce
Steps to reproduce the behavior:

• Call list workspaces api in different tenants

Code Snippet

SDK call: databricksManager.workspaces().list()

HTTP call: https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Databricks/workspaces?api-version=2021-04-01-preview

Expected behavior
Expected that API would send the same response for the same API call.

Screenshots

Setup (please complete the following information):

• OS: [e.g. iOS] - Ubuntu
• IDE: [e.g. IntelliJ] - N/A
• Library/Libraries: [com.azure.resourcemanager.azure-resourcemanager-databricks:1.0.0-beta.3]
• Java version: 11
• App Server/Environment: Tomcat
• Frameworks: Spring Boot

[weidongxu-microsoft]

@costichiulan

Do you mean, SDK is same version, calling on a same API, but on one tenant, the response body is of an unexpected shape (an array, with is different from that of other tenants)?

If so, we will need to connect you with backend dev. This is not an SDK issue.

[weidongxu-microsoft]

databricks got a few newer stable api-versions
https://github.com/Azure/azure-rest-api-specs/tree/main/specification/databricks/resource-manager/Microsoft.Databricks/stable/2023-02-01

Maybe we can release a beta-4 on 2023-02-01 next month, to see if they've fixed on newer api-version.

[costichiulan]

@costichiulan

Do you mean, SDK is same version, calling on a same API, but on one tenant, the response body is of an unexpected shape (an array, with is different from that of other tenants)?

If so, we will need to connect you with backend dev. This is not an SDK issue.

@weidongxu-microsoft Yes, it's not an issue with the SDK, but with microsoft azure resource API ( databricks ) , because it responds in a different way for same payload. You can close this issue though. If you can connect me with someone who works on Azure Resource API ( databricks ) it would be great. Thanks.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @arindamc.

[weidongxu-microsoft]

@arindamc

Customer facing this issue on 2021-04-01-preview. Would you help check on backend?

SDK will try a new stable api-version of 2023-05-01.

[weidongxu-microsoft]

@costichiulan

SDK released 1.0.0-beta.4 on their latest stable api-version 2023-05-01
https://central.sonatype.com/artifact/com.azure.resourcemanager/azure-resourcemanager-databricks/1.0.0-beta.4

You may try it on that tenant, to see if this issue been fixed in backend's latest api-version.

[weidongxu-microsoft]

We got feedback from backend.

Our team already this issue and fix is in rollout, it should reach all the regions by end of week.

[costichiulan]

Thanks for help !

…

On 14 Jun 2023, at 06:44, Weidong Xu ***@***.***> wrote: We got feedback from backend. Our team already this issue and fix is in rollout, it should reach all the regions by end of week.

[github-actions]

Hi @costichiulan. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[costichiulan]

@XiaofeiCao issue was fixed. Thanks