Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SourceDefinition.meta should represent merged base meta and table_meta #9766

Closed
1 task done
Tracked by #8917
MichelleArk opened this issue Mar 15, 2024 · 0 comments · Fixed by #9723
Closed
1 task done
Tracked by #8917

SourceDefinition.meta should represent merged base meta and table_meta #9766

MichelleArk opened this issue Mar 15, 2024 · 0 comments · Fixed by #9723
Assignees

Comments

@MichelleArk
Copy link
Contributor

Housekeeping

  • I am a maintainer of dbt-core

Short description

As part of the adapter decoupling work, we removed the BaseRelation.create_from_source and BaseRelation.create_from_node methods in favor of a consolidated BaseRelation.create_from method that accepts quoting: HasQuoting and relation_config: RelationConfig (#9210).

RelationConfig includes meta, as this is a common attribute shared across most node types & sources. However, for adapters that were relying on accessing source_meta to get the source-level metadata (e.g. duckdb), meta is not sufficient because it only includes the table meta, and uses source_meta to store the source-level meta property

Acceptance criteria

In dbt-core, update the meta field on SourceDefinition to include fields from source_meta by default.

This should be a backward compatible change if table_meta takes precedence over source_meta, which would be expected by dbt's typical config handling.

Suggested Tests

  1. SourceDefinition parsing should apply the appropriate precedence between table & source meta as described in AC

Impact to Other Teams

None

Will backports be required?

No

Context

Alternative options considered here: duckdb/dbt-duckdb#342 (comment)

@MichelleArk MichelleArk added user docs [docs.getdbt.com] Needs better documentation and removed user docs [docs.getdbt.com] Needs better documentation labels Mar 15, 2024
@MichelleArk MichelleArk self-assigned this Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant