feat: add enforcement for invariants to commit checker #1882
Conversation
|
ACTION NEEDED delta-rs follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
hntd187
changed the title
rtyler
changed the title
There was a problem hiding this comment.
There may have been a bit of miscommunication :).
The thing we need to decide is, if we do the invariant check or not - i.e. are invariants enabled. This is the case for 2 <= writer_version < 7 and if writer_version == 7 & 'invariants' in writer_features. And only if invariants are enabled we should inspect the schema to see if there are any invariants to validate.
One way to do this would be to make sure we only pass invariants to the DeltaDataChecker if they are enabled.
It also means we likely need to error if invariants are enabled and we have some defined, but the datafusion features and with that the checker is not available.
|
According to the protocol invariants for 2<=>6 are always enabled, so you'd only care to check if you were 7 or higher. Are you able to disable them prior to writer feature checks? |
No, as you said, between 2 and 6 they are always enabled. The main point is that it is not an error if they are configured on the schema, so we should not error, rather if if they are not enabled, we just kind of pretend they don't exist and do nothing w.r.t. to them ... |
|
I added this in check constraints anyways so this PR isn't needed anymore. |
Description
Added invariant checking in the commit checker.
Related Issue(s)
#1568 closes #1880
Documentation