Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Finish Constraint Support for Spark #747

Merged
merged 9 commits into from
May 24, 2023

Conversation

peterallenwebb
Copy link
Contributor

@peterallenwebb peterallenwebb commented Apr 27, 2023

resolves #656, resolves #657

Description

Add model-level constraint support to Spark

Checklist

@cla-bot cla-bot bot added the cla:yes label Apr 27, 2023
@peterallenwebb peterallenwebb marked this pull request as ready for review May 23, 2023 20:16
@peterallenwebb peterallenwebb requested a review from a team as a code owner May 23, 2023 20:16
"violate the new NOT NULL constraint",
"(id > 0) violated by row with values:", # incremental mats
"DELTA_VIOLATE_CONSTRAINT_WITH_VALUES", # incremental mats
"NOT NULL constraint violated for column",
]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new error messages were needed here, as it appears the messages coming from Databricks have changed slightly, and some of the messages were also truncated differently.

@dbeatty10
Copy link
Contributor

dbeatty10 commented May 24, 2023

Heads-up @peterallenwebb: the pull request description needs to be something like this to close multiple issues:

resolves #656, resolves #655

Otherwise, it will only auto-close the first issue, and the subsequent issue(s) will be left open.

@dbeatty10
Copy link
Contributor

@peterallenwebb Nevermind my comment about syntax for resolves since it looks like #655 is already closed.

Copy link
Contributor

@McKnight-42 McKnight-42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me

@dbeatty10
Copy link
Contributor

@peterallenwebb nevermind my nevermind 😅 -- from the changelog entry, it appears that you meant #657 instead of #655?

In that case, the PR description should be updated to:

resolves #656, resolves #657

Copy link
Contributor

@dbeatty10 dbeatty10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting an update to the changelog entry.

.changes/unreleased/Features-20230427-123135.yaml Outdated Show resolved Hide resolved
@peterallenwebb peterallenwebb dismissed dbeatty10’s stale review May 24, 2023 15:10

Made change as requested.

@peterallenwebb peterallenwebb merged commit 4f1039d into main May 24, 2023
@peterallenwebb peterallenwebb deleted the paw/CT-2223-model-level-constraints branch May 24, 2023 17:41
github-actions bot pushed a commit that referenced this pull request Jun 5, 2023
* CT-2223: Add CONSTRAINT_SUPPORT mapping to adapter implementation

* CT-2223: Model constraints implemented for spark

* CT-2223: Add unit test

* CT-2223: Exclude test not applicable to spark

* CT-2223: Another unit test fix.

* CT-2223: Final (?) unit test fix

* CT-2223: Final (??) unit test fix

* Update .changes/unreleased/Features-20230427-123135.yaml

Co-authored-by: Doug Beatty <[email protected]>

---------

Co-authored-by: Doug Beatty <[email protected]>
(cherry picked from commit 4f1039d)
MichelleArk pushed a commit that referenced this pull request Jun 6, 2023
* CT-2223: Add CONSTRAINT_SUPPORT mapping to adapter implementation

* CT-2223: Model constraints implemented for spark

* CT-2223: Add unit test

* CT-2223: Exclude test not applicable to spark

* CT-2223: Another unit test fix.

* CT-2223: Final (?) unit test fix

* CT-2223: Final (??) unit test fix

* Update .changes/unreleased/Features-20230427-123135.yaml

Co-authored-by: Doug Beatty <[email protected]>

---------

Co-authored-by: Doug Beatty <[email protected]>
(cherry picked from commit 4f1039d)

Co-authored-by: Peter Webb <[email protected]>
@colin-rogers-dbt colin-rogers-dbt mentioned this pull request Jun 20, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
4 participants