databricks: switching `parameters` to be a proper struct #6917

tombuildsstuff · 2019-08-09T11:47:12Z

As a part of the provisioning process, Databricks Clusters allow users to override certain values via a parameters block. Since there appears to be a fixed list of these:

customPrivateSubnetName, customPublicSubnetName, customVirtualNetworkId, enableNoPublicIp, relayNamespaceName, storageAccountName, storageAccountSkuName, vnetAddressPrefix, resourceTags, amlWorkspaceId.

from a usability perspective this would match other SDK's if these values were exposed as a native struct/class depending on the language, rather than being a generic dictionary; as such this PR changes parameters to a struct rather than a dictionary.

Latest improvements:

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Contribution checklist:

I have reviewed the documentation for the workflow.
Validation tools were run on swagger spec(s) and have all been fixed in this PR.
The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

AutorestCI · 2019-08-09T11:47:53Z

Automation for azure-sdk-for-go

Encountered an unknown error: (azure-sdk-for-go)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout cbfa1c61556efb14c8a2089439fd410ace3d3623
  stderr: 'fatal: reference is not a tree: cbfa1c61556efb14c8a2089439fd410ace3d3623'

AutorestCI · 2019-08-09T11:48:13Z

Automation for azure-sdk-for-python

Encountered an unknown error: (azure-sdk-for-python)

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 33, in exception_to_github
    yield context
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 170, in rest_handle_action
    return rest_pull_close(body, restapi_repo, sdk_pr_target_repo, sdkbase, sdk_tag)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github.py", line 185, in rest_pull_close
    rest_pr_management(rest_pr, sdk_pr_target_repo, sdk_tag, sdk_default_base)
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/restapi/github_handler.py", line 151, in rest_pr_management
    sdk_tag=sdk_tag
  File "/usr/local/lib/python3.6/dist-packages/swaggertosdk/SwaggerToSdkNewCLI.py", line 254, in generate_sdk_from_git_object
    with manage_git_folder(gh_token, Path(temp_dir) / Path("rest"), branched_rest_api_id, pr_number=pr_number) as restapi_git_folder, \
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 272, in manage_git_folder
    clone_to_path(gh_token, temp_dir, split_git_id[0], branch_or_commit=branch, pr_number=pr_number)
  File "/usr/local/lib/python3.6/dist-packages/azure_devtools/ci_tools/github_tools.py", line 212, in clone_to_path
    repo.git.checkout(branch_or_commit)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 548, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 1014, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/git/cmd.py", line 825, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git checkout cbfa1c61556efb14c8a2089439fd410ace3d3623
  stderr: 'fatal: reference is not a tree: cbfa1c61556efb14c8a2089439fd410ace3d3623'

azuresdkci · 2019-08-09T12:08:45Z

Can one of the admins verify this patch?

Juliehzl · 2019-08-09T14:35:01Z

Pls have a look at CI error. I think you should update your examples for the new swagger parameters.

ryansbenson

Approved from ARMs side

Juliehzl · 2019-08-15T05:13:39Z

@tombuildsstuff Please add body part for 201 response in workspacecreate example. If you want to return empty body, you sholud use "body": "" not {}.

yungezz · 2019-08-28T08:43:41Z

hi @tombuildsstuff @Juliehzl what's the conclusion on this PR? is it ready for merge?

openapi-sdkautomation · 2019-09-11T02:57:20Z

In Testing, Please Ignore

[Logs] (Generated from a6735f7, Iteration 3)

Python: test-repo-billy/azure-sdk-for-python [Logs] [Diff]

azure-mgmt-databricks [Logs] [Instructions] [Generation PR]
- azure-mgmt-databricks-0.1.0.zip
- azure_mgmt_databricks-0.1.0-py2.py3-none-any.whl

Go: test-repo-billy/azure-sdk-for-go [Logs] [Diff]

databricks/mgmt/2018-04-01 [Logs] [Instructions] [Generation PR]
- databricks.mgmt.2018-04-01.zip

JavaScript: test-repo-billy/azure-sdk-for-js [Logs] [Diff]

@azure/arm-databricks [Logs] [Instructions] [Generation PR]
- azure-arm-databricks-1.1.0.tgz

Juliehzl · 2019-09-11T03:02:33Z

@tombuildsstuff Can you try to make CI pass according log information?

Juliehzl · 2019-09-21T14:20:53Z

Looks good in general. @tombuildsstuff can you make CI pass first?

tombuildsstuff · 2019-10-31T16:47:36Z

@Juliehzl sure, will try and take a look today/tomorrow

Juliehzl · 2019-11-12T01:43:43Z

@Juliehzl sure, will try and take a look today/tomorrow

Any update?

Juliehzl · 2019-11-22T05:21:50Z

@tombuildsstuff Do you have any update for this PR?

Juliehzl · 2019-11-27T02:45:24Z

LGTM for specification side. But no response from @tombuildsstuff to make CI pass. In this way, I will close this PR and @tombuildsstuff when you are ready, feel free to reopen it.

databricks: switching parameters to be a proper struct

2f91df5

azuresdkci assigned Juliehzl Aug 9, 2019

azuresdkci requested a review from Juliehzl August 9, 2019 11:48

tombuildsstuff added 2 commits August 9, 2019 12:49

removing the unused required field

0e620d2

linting

a6735f7

This was referenced Aug 9, 2019

[AutoPR databricks/resource-manager] databricks: switching parameters to be a proper struct Azure/azure-sdk-for-python#6743

Closed

[AutoPR databricks/resource-manager] databricks: switching parameters to be a proper struct Azure/azure-sdk-for-go#5482

Closed

tombuildsstuff mentioned this pull request Aug 9, 2019

r/databricks_workspace: support for workspace parameters hashicorp/terraform-provider-azurerm#3889

Merged

Juliehzl added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Aug 9, 2019

ryansbenson reviewed Aug 14, 2019

View reviewed changes

ryansbenson added ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review and removed WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required labels Aug 14, 2019

PhoenixHe-NV mentioned this pull request Aug 16, 2019

[Mirror] databricks: switching parameters to be a proper struct test-repo-tih/azure-rest-api-specs#117

Closed

PhoenixHe-NV closed this Sep 11, 2019

PhoenixHe-NV reopened this Sep 11, 2019

AutorestCI mentioned this pull request Sep 11, 2019

[AutoPR databricks/resource-manager] databricks: switching parameters to be a proper struct Azure/azure-sdk-for-go#5742

Closed

Juliehzl added the Reviewed-ChangesRequested <valid label in PR review process>add this label when assignee request changes after review label Sep 23, 2019

tombuildsstuff mentioned this pull request Sep 25, 2019

Open Source the Wiki required for contributing to API Specifications #7315

Closed

Juliehzl closed this Oct 17, 2019

Juliehzl reopened this Oct 17, 2019

tombuildsstuff force-pushed the databricks-parameters-struct branch 3 times, most recently from 69a0931 to 8ec5de2 Compare November 4, 2019 12:00

AutorestCI mentioned this pull request Nov 4, 2019

[AutoPR databricks/resource-manager] databricks: switching parameters to be a proper struct Azure/azure-sdk-for-go#6248

Closed

DataBricks: matching the Swagger to the API request/response

8c36be0

tombuildsstuff force-pushed the databricks-parameters-struct branch from 8ec5de2 to 8c36be0 Compare November 4, 2019 14:13

Juliehzl approved these changes Nov 27, 2019

View reviewed changes

Juliehzl closed this Nov 27, 2019

mbfrahry mentioned this pull request Dec 2, 2019

databricks: switching parameters to be a proper struct #7872

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

databricks: switching `parameters` to be a proper struct #6917

databricks: switching `parameters` to be a proper struct #6917

tombuildsstuff commented Aug 9, 2019

AutorestCI commented Aug 9, 2019 •

edited

Loading

AutorestCI commented Aug 9, 2019 •

edited

Loading

azuresdkci commented Aug 9, 2019

Juliehzl commented Aug 9, 2019

ryansbenson left a comment

Juliehzl commented Aug 15, 2019

yungezz commented Aug 28, 2019

openapi-sdkautomation bot commented Sep 11, 2019 •

edited

Loading

Juliehzl commented Sep 11, 2019

Juliehzl commented Sep 21, 2019

tombuildsstuff commented Oct 31, 2019

Juliehzl commented Nov 12, 2019

Juliehzl commented Nov 22, 2019

Juliehzl commented Nov 27, 2019

databricks: switching parameters to be a proper struct #6917

databricks: switching parameters to be a proper struct #6917

Conversation

tombuildsstuff commented Aug 9, 2019

Latest improvements:

Contribution checklist:

ARM API Review Checklist

AutorestCI commented Aug 9, 2019 • edited Loading

Automation for azure-sdk-for-go

AutorestCI commented Aug 9, 2019 • edited Loading

Automation for azure-sdk-for-python

azuresdkci commented Aug 9, 2019

Juliehzl commented Aug 9, 2019

ryansbenson left a comment

Choose a reason for hiding this comment

Juliehzl commented Aug 15, 2019

yungezz commented Aug 28, 2019

openapi-sdkautomation bot commented Sep 11, 2019 • edited Loading

In Testing, Please Ignore

[Logs] (Generated from a6735f7, Iteration 3)

Juliehzl commented Sep 11, 2019

Juliehzl commented Sep 21, 2019

tombuildsstuff commented Oct 31, 2019

Juliehzl commented Nov 12, 2019

Juliehzl commented Nov 22, 2019

Juliehzl commented Nov 27, 2019

databricks: switching `parameters` to be a proper struct #6917

databricks: switching `parameters` to be a proper struct #6917

AutorestCI commented Aug 9, 2019 •

edited

Loading

AutorestCI commented Aug 9, 2019 •

edited

Loading

openapi-sdkautomation bot commented Sep 11, 2019 •

edited

Loading