dbt-core v1.8.0

dbt-core 1.8.0 - Julian Abele (May 09, 2024)

Breaking Changes

• Remove adapter.get_compiler interface (#9148)
• Move AdapterLogger to adapters folder (#9151)
• Rm --dry-run flag from 'dbt deps --add-package', in favor of just 'dbt deps --lock' (#9100)
• move event manager setup back to core, remove ref to global EVENT_MANAGER and clean up event manager functions (#9150)
• Remove dbt-tests-adapter and dbt-postgres packages from dbt-core (#9455)
• Update the default behaviour of require_explicit_package_overrides_for_builtin_materializations to True. (#10062)

Features

• Initial implementation of unit testing (#8287)
• Unit test manifest artifacts and selection (#8295)
• Support config with tags & meta for unit tests (#8294)
• Allow adapters to include package logs in dbt standard logging (#7859)
• Enable inline csv fixtures in unit tests (#8626)
• Add drop_schema_named macro (#8025)
• migrate utils to common and adapters folders (#8924)
• Move Agate helper client into common (#8926)
• remove usage of dbt.config.PartialProject from dbt/adapters (#8928)
• Add exports to SavedQuery spec (#8892)
• Support unit testing incremental models (#8422)
• Add support of csv file fixtures to unit testing (#8290)
• Remove legacy logger (#8027)
• Unit tests support --defer and state:modified (#8517)
• Support setting export configs hierarchically via saved query and project configs (#8956)
• Support source inputs in unit tests (#8507)
• Use daff to render diff displayed in stdout when unit test fails (#8558)
• Global config for --target and --profile CLI flags and DBT_TARGET and DBT_PROFILE environment variables. (#7798)
• Move unit testing to test command (#8979)
• Support --empty flag for schema-only dry runs (#8971)
• Support unit tests in non-root packages (#8285)
• Convert the tests config to data_tests in both dbt_project.yml and schema files. in schema files. (#8699)
• Make fixture files full-fledged parts of the manifest and enable partial parsing (#9067)
• Adds support for parsing conversion metric related properties for the semantic layer. (#9203)
• Package selector syntax for the current package (#6891)
• In build command run unit tests before models (#9128)
• Move flags from UserConfig in profiles.yml to flags in dbt_project.yml (#9183)
• Added hook support for dbt source freshness (#5609)
• Align with order of unit test output when actual differs from expected (#9370)
• Added support for external nodes in unit test nodes (#8944)
• Enable unit testing versioned models (#9344)
• Enable list command for unit tests (#8508)
• Integration Test Optimizations (#9498)
• Accelerate integration tests with caching. (#9498)
• Cache environment variables (#9489)
• Support meta at the config level for Metric nodes (#9441)
• Add cache to SavedQuery config (#9540)
• Support scrubbing secret vars (#7247)
• Allow excluding resource types for build, list, and clone commands, and provide env vars (#9237)
• SourceDefinition.meta represents source-level and table-level meta properties, instead of only table-level (#9766)
• Allow metrics in semantic layer filters. (#9804)
• Add wildcard support to the group selector method (#9811)
• source freshness precomputes metadata-based freshness in batch, if possible (#8705)
• Better error message when trying to select a disabled model (#9747)
• Support SQL in unit testing fixtures (#9405)
• Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages (#10007)
• add --empty flag to dbt build command (#10026)
• Ability to silence warnings via warn_error_options (#9644)
• Allow aliases error for include and warn for exclude in warn_error_options (#9644)
• Add unit_test: selection method (#10053)

Fixes

• For packages installed with tarball method, fetch metadata to resolve nested dependencies (#8621)
• Fix partial parsing not working for semantic model change (#8859)
• Handle unknown type_code for model contracts (#8877, #8353)
• Rework get_catalog implementation to retain previous adapter interface semantics (#8846)
• Add back contract enforcement for temporary tables on postgres (#8857)
• Add version to fqn when version==0 (#8836)
• Fix cased comparison in catalog-retrieval function. (#8939)
• Catalog queries now assign the correct type to materialized views (#8864)
• Fix compilation exception running empty seed file and support new Integer agate data_type (#8895)
• Make relation filtering None-tolerant for maximal flexibility across adapters. (#8974)
• Update run_results.json from previous versions of dbt to support deferral and rerun from failure (#9010)
• Use MANIFEST.in to recursively include all jinja templates; fixes issue where some templates were not included in the distribution (#9016)
• Fix git repository with subdirectory for Deps (#9000)
• Use seed file from disk for unit testing if rows not specified in YAML config (#8652)
• Fix formatting of tarball information in packages-lock.yml (#9062)
• deps: Lock git packages to commit SHA during resolution (#9050)
• deps: Use PackageRenderer to read package-lock.json (#9127)
• Ensure we produce valid jsonschema schemas for manifest, catalog, run-results, and sources (#8991)
• Get sources working again in dbt docs generate (#9119)
• Fix parsing f-strings in python models (#6976)
• Preserve the value of vars and the --full-refresh flags when using retry. (#9112)
• fix lock-file bad indentation (#9319)
• fix configuration of turning test warnings into failures with WARN_ERROR_OPTIONS (#7761)
• Support reasonably long unit test names (#9015)
• Fix back-compat pa...

dbt-core v1.8.0rc2

dbt-core 1.8.0-rc2 - May 08, 2024

Under the Hood

• Enable use of context in serialization (#10093)

dbt-core v1.8.0rc1

dbt-core 1.8.0-rc1 - May 02, 2024

Breaking Changes

• Update the default behaviour of require_explicit_package_overrides_for_builtin_materializations to True. (#10062)

Features

• Package selector syntax for the current package (#6891)
• Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages (#10007)
• add --empty flag to dbt build command (#10026)
• Ability to silence warnings via warn_error_options (#9644)
• Allow aliases error for include and warn for exclude in warn_error_options (#9644)

Fixes

• Add NodeRelation to SavedQuery Export (#9534)
• Use consistent secret scrubbing with the log function. (#9987)
• Validate against empty strings in package definitions (#9985)
• Fix default value for indirect selection in selector cannot overwritten by CLI flag and env var (#9976, #7673)
• Simplify error message if test severity isn't 'warn' or 'error' (#9715)
• Support overriding source level loaded_at_field with a null table level definition (#9320)

Docs

• Enable display of unit tests (dbt-docs/#501)
• Unit tests not rendering (dbt-docs/#506)

Under the Hood

• Split up deferral across parsing (adding 'defer_relation' from state manifest) and runtime ref resolution" (#9199)
• Raise deprecation warning if installed package overrides built-in materialization (#9971)
• Use the SECRET_ENV_PREFIX from dbt_common instead of duplicating it in dbt-core (#10018)
• Enable use of record mode via environment variable (#10045)
• Consistent naming + deprecation warnings for "legacy behavior" flags (#10062)

Dependencies

• Update the agate pin to "agate>=1.7.0,<1.10" (#9934)

Contributors

• @barton996 (#6891)
• @mjkanji (#9934)

dbt-core v1.7.14

dbt-core 1.7.14 - May 02, 2024

Features

• Move flags from UserConfig in profiles.yml to flags in dbt_project.yml (#9183)
• Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages (#10007)

Fixes

• remove materialized views from renambeable relation and remove a quote (#127)
• Replace usage of Set with List to fix issue with index updates intermittently happening out of order (#72)

Under the Hood

• Raise deprecation warning if installed package overrides built-in materialization (#9971)
• Remove the final underscore from secret environment variable constants. (#10052)

dbt-core v1.6.14

dbt-core 1.6.14 - May 02, 2024

Features

• Move flags from UserConfig in profiles.yml to flags in dbt_project.yml (#9183)
• Add require_explicit_package_overrides_for_builtin_materializations to dbt_project.yml flags, which can be used to opt-out of overriding built-in materializations from packages (#10007)

Fixes

• Replace usage of Set with List to fix issue with index updates intermittently happening out of order (#72)

Under the Hood

• Raise deprecation warning if installed package overrides built-in materialization (#9971)
• Remove the final underscore from secret environment variable constants. (#10052)

dbt-core v1.8.0b3

dbt-core 1.8.0-b3 - April 18, 2024

Features

• Support scrubbing secret vars (#7247)
• Add wildcard support to the group selector method (#9811)
• source freshness precomputes metadata-based freshness in batch, if possible (#8705)
• Better error message when trying to select a disabled model (#9747)
• Support SQL in unit testing fixtures (#9405)

Fixes

• fix configuration of turning test warnings into failures with WARN_ERROR_OPTIONS (#7761)
• Fix conflict with newer versions of Snowplow tracker (#8719)
• Only create the packages-install-path / dbt_packages folder during dbt deps (#6985, #9584)
• Exclude password-like fields for considering reparse (#9795)
• Fixed query comments test (#9860)
• Begin warning people about spaces in model names (#9397)
• Disambiguiate FreshnessConfigProblem error message (#9891)

Under the Hood

• Remove non dbt.artifacts dbt.* imports from dbt/artifacts (#9926)
• Migrate to using error_tag provided by dbt-common (#9914)
• Add a test for semantic manifest and move test fixtures needed for it (#9665)

Dependencies

• Relax pathspec upper bound version restriction (#9373)
• Bump python from 3.10.7-slim-nullseye to 3.11.2-slim-bullseye in /docker (#9687)
• Remove duplicate dependency of protobuf in dev-requirements (#9830)
• Bump black from 23.3.0 to >=24.3.0,<25.0 (#8074)

Security

• Bump sqlparse to >=0.5.0, <0.6.0 to address GHSA-2m57-hf25-phgg (#9951)

Contributors

• @SamuelBFavarin (#9747)
• @akurdyukov (#8719)
• @damian3031 (#9860)
• @edgarrmondragon (#8719)
• @emmoop (#9951)
• @heysweet (#9811)
• @jx2lee (#7761)
• @nielspardon (#7247)
• @niteshy (#9830)
• @rzjfr (#9373)

dbt-core v1.7.13

dbt-core 1.7.13 - April 18, 2024

Security

• Bump sqlparse to >=0.5.0, <0.6.0 to address GHSA-2m57-hf25-phgg (#9951)

Contributors

• @emmoop (#9951)

dbt-core v1.6.13

dbt-core 1.6.13 - April 18, 2024

Security

• Bump sqlparse to >=0.5.0, <0.6.0 to address GHSA-2m57-hf25-phgg (#9951)

Contributors

• @emmoop (#9951)

dbt-core v1.7.12

dbt-core 1.7.12 - April 16, 2024

Fixes

• Fix assorted source freshness edgecases so check is run or actionable information is given (#9078)
• Exclude password-like fields for considering reparse (#9795)

dbt-core v1.6.12

dbt-core 1.6.12 - April 16, 2024

Fixes

• Exclude password-like fields for considering reparse (#9795)