Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: sql for missing columns and mysql based dialects (amundsen#550) #305

Merged
merged 1 commit into from
Jul 20, 2020

Conversation

pPanda-beta
Copy link
Contributor

@pPanda-beta pPanda-beta commented Jul 19, 2020

This solves amundsen-io/amundsen#550

Summary of Changes

  1. Added missing column "database" since HiveWatermark is not used anymore
  2. Put both "database" and "schema" column names under blockquotes, as those are reserved keywords

Tests

Example code, no tests. Manually verify by running with docker-compose.

Documentation

CheckList

Make sure you have checked all steps below to ensure a timely review.

  • PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
  • PR includes a summary of changes.
  • PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
  • In case of new functionality, my PR adds documentation that describes how to use it.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
  • PR passes make test

@pPanda-beta pPanda-beta changed the title Fix sql for missing columns and mysql based dialects (amundsen#550) fix: sql for missing columns and mysql based dialects (amundsen#550) Jul 19, 2020
@codecov-commenter
Copy link

Codecov Report

Merging #305 into master will decrease coverage by 0.40%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #305      +/-   ##
==========================================
- Coverage   74.30%   73.90%   -0.41%     
==========================================
  Files         105      105              
  Lines        4492     4492              
  Branches      419      419              
==========================================
- Hits         3338     3320      -18     
- Misses       1049     1058       +9     
- Partials      105      114       +9     
Impacted Files Coverage Δ
...ilder/transformer/regex_str_replace_transformer.py 89.28% <0.00%> (-7.15%) ⬇️
databuilder/loader/file_system_neo4j_csv_loader.py 83.33% <0.00%> (-5.96%) ⬇️
...abuilder/extractor/snowflake_metadata_extractor.py 90.62% <0.00%> (-4.69%) ⬇️
...tabuilder/extractor/postgres_metadata_extractor.py 91.93% <0.00%> (-3.23%) ⬇️
databuilder/extractor/mssql_metadata_extractor.py 92.53% <0.00%> (-2.99%) ⬇️
...tor/dashboard/redash/redash_dashboard_extractor.py 91.50% <0.00%> (-0.95%) ⬇️
databuilder/publisher/neo4j_csv_publisher.py 83.49% <0.00%> (-0.95%) ⬇️
databuilder/task/neo4j_staleness_removal_task.py 84.16% <0.00%> (-0.84%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24b3b0a...9faf470. Read the comment docs.

@feng-tao feng-tao merged commit 4b7b147 into amundsen-io:master Jul 20, 2020
@pPanda-beta pPanda-beta deleted the patch-1 branch July 20, 2020 21:36
ccarterlandis pushed a commit to Gusto/amundsendatabuilder_contrib that referenced this pull request Aug 14, 2020
jerryzhu2007 pushed a commit to kylg/amundsendatabuilder that referenced this pull request Aug 20, 2020
* commit 'e14b33e776929f8b020f1c6fec75d0fb83687693': (23 commits)
  Fix Athena sample DAG (amundsen-io#341)
  fix: Update postgres_sample_dag to set table extract job as upstream for elastic search publisher (amundsen-io#340)
  chore: mypy cleanup (convert last comment types, remove noqa imports) (amundsen-io#338)
  chore: Convert typings to mypy (amundsen-io#311)
  chore: replace all references of Lyft repo with Amundsen (amundsen-io#323)
  feat: add github actions for databuilder (amundsen-io#336)
  build: fix broken tests in Python 3.7, test in CI (amundsen-io#334)
  fix(deps): Unpin attrs (amundsen-io#332)
  ci: add dependabot config (amundsen-io#330)
  Change repo name in travis file (amundsen-io#324)
  tests: add mock for bigquery auth (amundsen-io#313)
  feat: allow hive sql to be provided as config (amundsen-io#312)
  chore: remove python2 (amundsen-io#310)
  chore: update deps for databuilder (amundsen-io#309)
  fix: cypher statement param issue in Neo4jStalenessRemovalTask (amundsen-io#307)
  fix: Added missing job tag key in hive_sample_dag.py (amundsen-io#308)
  feat: enhance glue extractor (amundsen-io#306)
  fix: Fix sql for missing columns and mysql based dialects (#550) (amundsen-io#305)
  docs: Fix broken doc link to dashboard_execution model (amundsen-io#296)
  chore: apply license headers to all the source files (amundsen-io#304)
  ...

# Conflicts:
#	README.md
#	databuilder/extractor/kafka_source_extractor.py
#	databuilder/publisher/neo4j_csv_publisher.py
#	docs/models.md
#	example/scripts/sample_data_loader.py
#	setup.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants