Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2126] [Spike] Enforce model contracts for python models #6984

Closed
Tracked by #7979 ...
MichelleArk opened this issue Feb 15, 2023 · 3 comments
Closed
Tracked by #7979 ...

[CT-2126] [Spike] Enforce model contracts for python models #6984

MichelleArk opened this issue Feb 15, 2023 · 3 comments
Assignees
Labels

Comments

@MichelleArk
Copy link
Contributor

If possible, enforce model contracts for python models.

Otherwise, keep the parsing error we're currently if contracts are used on python models around.

Relevant discussion: https://github.com/dbt-labs/dbt-core/pull/6271/files#r1084049809

@github-actions github-actions bot changed the title Enforce model contracts for python models [CT-2126] Enforce model contracts for python models Feb 15, 2023
@MichelleArk MichelleArk changed the title [CT-2126] Enforce model contracts for python models [CT-2126] [Spike] Enforce model contracts for python models Feb 23, 2023
@jtcohen6 jtcohen6 added the spike label Feb 28, 2023
@jtcohen6
Copy link
Contributor

jtcohen6 commented Jul 5, 2023

For SQL models, we run an "empty" version of the model query (where false limit 0) to extract the output schema in advance. I don't think any equivalent exists for Python models.

We could have the option of:

  • Always create the model in a temporary location
  • Compare the output result to the contracted set of columns, and raise an error if they differ
  • Apply constraints after-the-fact, and raise an error if their enforcement fails
  • IFF the contract enforcement passes, atomically replace the existing table with the current version of the model (whether alter swap, or clone, or create or replace {tbl} as select * from {tmp_tbl})

The biggest downsides are:

  • (For end users) Contract enforcement will take much more time & compute, versus for SQL models
  • (For us & adapter maintainers) This would require slightly different & trickier codepaths for contract enforcement, within create_table_as and py_write_table

Copy link
Contributor

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

@github-actions github-actions bot added the stale Issues that have gone stale label Jan 24, 2024
Copy link
Contributor

github-actions bot commented Feb 2, 2024

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants