Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "partitions" to the adapter specific config (#2256) #2280

Merged
merged 1 commit into from
Apr 1, 2020

Conversation

beckjake
Copy link
Contributor

resolves #2256

Description

Add partitions to the BigQuery adapter-specific config, and test that the config works.

Checklist

  • I have signed the CLA
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • I have updated the CHANGELOG.md and added information about my change to the "dbt next" section.

@cla-bot cla-bot bot added the cla:yes label Mar 31, 2020
@beckjake beckjake force-pushed the fix/add-partitions-bq-config branch from 7290d66 to c63f60e Compare April 1, 2020 16:06
@beckjake beckjake requested review from drewbanin and jtcohen6 and removed request for drewbanin April 1, 2020 16:29
Copy link
Contributor

@jtcohen6 jtcohen6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

There's two different uses for the partitions config in dbt-bigquery right now:

  • Creating ingestion-time partitioned tables, added in 0.10.0
  • An optional behavior of the insert_overwrite incremental strategy, added in 0.16.0

The tests added in this PR are largely checking the first feature, which we don't generally recommend in our modeling anymore. BigQuery's recent development has prioritized column partitioned tables over ingestion-time partitioned tables. That said, dbt has supported creating date-partitioned tables for several years, and unless we're about to stop supporting it, adding tests is A Good Thing.

@beckjake
Copy link
Contributor Author

beckjake commented Apr 1, 2020

The key of the test here is "does partitions make it through the configs", so I'm going to merge this! But that is good to know - do we have any tests of the insert_overwrite aspect of this config setting? If not, we should!

@beckjake beckjake merged commit ff0e955 into dev/0.16.1 Apr 1, 2020
@beckjake beckjake deleted the fix/add-partitions-bq-config branch April 1, 2020 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants