Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bigquery Sink using depot #179

Merged
merged 12 commits into from
Aug 5, 2022
Merged

feat: bigquery Sink using depot #179

merged 12 commits into from
Aug 5, 2022

Conversation

lavkesh
Copy link
Member

@lavkesh lavkesh commented Jul 6, 2022

This PR

  1. Enhances Add support for JSON data for BQ sink #84
  2. Enhances Extract bigquery sink to depot #184

Changelog:

  1. Bigquery sink using depot
  2. Json support in bigquery sink
  3. Introduced new config INPUT_SCHEMA_DATA_TYPE
  4. Extracted instrumentation into depot

* feat: adding odpf/depot depedencies

* feat: add github maven repo for odpf/depot

* feat: add github auth

* chore: depenedencies

* chore: version bump

* chore: version bump

* refactor: move common code to seperate class
* chore: version bump

* docs: add docs for depot repo

* docs: add bigquery metrics
@ravisuhag ravisuhag changed the title Bigquery Sink using depot feat: bigquery Sink using depot Jul 6, 2022
lavkesh and others added 5 commits July 21, 2022 07:52
* feat: json support for bq sink

* feat: add/update depot configs in sinkFactoryUtils

* chore: version bump and removing json related configs

* feat: add json schema type

* fix: fix typo and handle json messages on log dlq writer (#180)

* fix: fix typo and handle json messages on log dlq writer

* test: add logDlqwriter unit test

* chore: version bump

Co-authored-by: kevin.bheda <[email protected]>
Co-authored-by: mayur.gubrele <[email protected]>
Co-authored-by: lavkesh <[email protected]>
Co-authored-by: jesrypandawa <[email protected]>
@ravisuhag ravisuhag merged commit ab4c39b into main Aug 5, 2022
@ravisuhag ravisuhag deleted the feature-sink-depot branch August 5, 2022 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants