Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎉 Destination Databricks: Support Azure storage #15140

Closed
abaerptc opened this issue Jul 29, 2022 · 4 comments · Fixed by #15329
Closed

🎉 Destination Databricks: Support Azure storage #15140

abaerptc opened this issue Jul 29, 2022 · 4 comments · Fixed by #15329
Labels
community connectors/destination/databricks lang/java team/destinations Destinations team's backlog type/enhancement New feature or request

Comments

@abaerptc
Copy link
Contributor

Tell us about the problem you're trying to solve

I'd like to use the Databricks destination with Azure storage.

Describe the solution you’d like

I'd like the option to specify an Azure storage location as an alternative to S3 storage.

Describe the alternative you’ve considered or used

None.

Additional context

None.

Are you willing to submit a PR?

Yes.
@abaerptc abaerptc added needs-triage type/enhancement New feature or request labels Jul 29, 2022
@abaerptc
Copy link
Contributor Author

I have a prototype written already and I will open a PR shortly for review.

@natalyjazzviolin natalyjazzviolin changed the title Support Azure storage for Databricks destination 🎉 Destination Databricks: Support Azure storage Aug 1, 2022
@natalyjazzviolin
Copy link
Contributor

@abaerptc thanks for opening this issue, looking forward to your PR!

@grishick grishick added the team/destinations Destinations team's backlog label Sep 27, 2022
@creativedutchmen
Copy link

Any updates on this? This PR would make the life of Databricks users on Azure so much easier - the current approaches of either using minio or csv files are both suboptimal. Being able to use a CDC source and dedupe straight from Airbyte with Databricks, and having the result as a delta table is a game changer.

@abaerptc
Copy link
Contributor Author

@marcosmarxm @tuliren Any update on getting this merged in? #15329

@abaerptc abaerptc removed their assignment Oct 10, 2022
@tuliren tuliren mentioned this issue Oct 15, 2022
Merged
14 tasks
tuliren added a commit that referenced this issue Oct 15, 2022
@abaerptc @marcosmarxm
@tuliren @octavia-squidington-iii
🎉 Destination Databricks: Support Azure storage (#15140) (#15329)
1b134d8 
* 🎉 Destination Databricks: Support Azure storage (#15140)

* Update docs and version.

* Revert unintentional change to S3 config parsing.

* Revert unnecessary additional table property.

* change spec.json

* Ignore azure integration test

* Add issue link

* auto-bump connector version

Co-authored-by: Marcos Marx <[email protected]>
Co-authored-by: marcosmarxm <[email protected]>
Co-authored-by: Liren Tu <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>
Merged
jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this issue Oct 31, 2022
@abaerptc @marcosmarxm
@tuliren @octavia-squidington-iii @jhammarstedt
🎉 Destination Databricks: Support Azure storage (airbytehq#15140) (ai…
573d5e6 
…rbytehq#15329)

* 🎉 Destination Databricks: Support Azure storage (airbytehq#15140)

* Update docs and version.

* Revert unintentional change to S3 config parsing.

* Revert unnecessary additional table property.

* change spec.json

* Ignore azure integration test

* Add issue link

* auto-bump connector version

Co-authored-by: Marcos Marx <[email protected]>
Co-authored-by: marcosmarxm <[email protected]>
Co-authored-by: Liren Tu <[email protected]>
Co-authored-by: Octavia Squidington III <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
community connectors/destination/databricks lang/java team/destinations Destinations team's backlog type/enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

🎉 Destination Databricks: Support Azure storage (#15140)
5 participants
@creativedutchmen @grishick @natalyjazzviolin @octavia-squidington-iii @abaerptc