Restore ability to utilize `updated_at` for check_cols snapshots #5077

dbeatty10 · 2022-04-15T16:38:23Z

Restore ability to configure and utilize updated_at for snapshots using the check_cols strategy

resolves #5076

Description

#5076 describes a breaking change, and this PR converts it to a non-breaking change in the most minimal way possible.

Details

Verified that the functional test fails without the code change but passes after the code change
The check_relations_equal method excluded columns beginning with dbt_, which were the exact columns we want to compare, so modification was necessary
To make it a little easier to see the difference between where the snapshot dates are being sourced from, the timestamp_col column uses dates in January of 2016 and the updated_at configuration uses dates in July of 2016

Testing approach

Create a table that represents the expected data after a series of snapshots
- Use dbt seed to create the expected relation (snapshot_check_cols_updated_at_expected)
Execute a series of snapshots to create the data
- Use a series of (3) dbt snapshot commands to create the actual relation (snapshot_check_cols_updated_at_actual)
- my_snapshot.sql contains logic that can switch between 3 different versions of the data
Compare the two relations for equality

The functional test is designed to make sure dbt_updated_at matches the timestamp expression that we passed in through --vars.

The only change within the 3rd version of the data deletes a single row. Because this test doesn't cover the invalidate_hard_deletes=True option, the 3rd version of the data makes no updates. However, this version is included to make it easier to test for the expected behavior when invalidating hard deletes. See "future ideas to consider" below.

Pros

No (known) breaking changes
Uses the newer functional test conventions
Enables inclusion of dbt_* columns within the comparison of relations
Establishes the expected result for the default option when invalidate_hard_deletes=False

Cons

Not expected to work with adapters other than postgres
Still not possible to specify a subset of columns to exclude from comparison
Includes a 3rd version of the data which is appears not utilized

Future ideas to consider

The timestamp to utilize for hard deletes as an optional opt-in configuration for snapshots
- This is the "lower lift" option described in #4347 by @jtcohen6
For functional tests, allow the inclusion/exclusion of a subset of columns to compare between two relations
Make including all columns the default and allow selective exclusion of columns beginning with dbt_ only for those tests that need it.

Checklist

I have signed the CLA
I have run this code in development and it appears to resolve the stated issue
This PR includes tests, or tests are not required/relevant for this PR
I have added information about my change to be included in the CHANGELOG.

…sing the check_cols strategy

cla-bot · 2022-04-15T16:38:25Z

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Doug Beatty.
This is most likely caused by a git client misconfiguration; please make sure to:

check if your git client is configured with an email to sign commits git config --list | grep email
If not, set it up using git config --global user.email [email protected]
Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

github-actions · 2022-04-15T16:38:40Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

jtcohen6 · 2022-04-15T17:12:47Z

@cla-bot check

cla-bot · 2022-04-15T17:12:52Z

The cla-bot has been summoned, and re-checked this pull request!

ChenyuLInx · 2022-04-15T18:12:55Z

The change looks good! Can we add a test for this case so we don't accidentally break this next time?

dbeatty10 · 2022-04-15T20:10:39Z

Will work on some test cases for this.

stu-k

Nothing stands out, though I'm not the most proficient in testing snapshots, so it might be a good idea to wait for one more approval 👍

ChenyuLInx

This looks great! Thanks @dbeatty10 for the fast fix and test!
Left one question/comment to make sure I understood your test correctly.

ChenyuLInx · 2022-04-20T20:47:13Z

tests/functional/simple_snapshot/test_check_cols_updated_at_snapshot.py

+    }
+
+
+def test_simple_snapshot(project):


Make sure I understand the test case correctly here.
dbt seed would create thesnapshot_check_cols_updated_at_expected table. All data in snapshot_check_cols_updated_at_actual is purely generated by the logic in the snapshot.sql.

And for the command run_dbt(["snapshot", "--vars", "{version: 3, updated_at: 2016-07-03}"]) command there's no data in snapshot table got generated because no data got modified.

This test is also designed to make sure the time for dbt_updated_at actually has the time that we passed in through '--vars'. Which is the main change this PR is doing

This is a great test! I would recommend adding a few short comments to document the intention of each command.

Thank you for this excellent feedback @ChenyuLInx!

You are correct in your understanding of the test case. I've updated the description of the pull request to reflect your feedback. Or were you thinking of adding these as comments within the code itself?

Added some comments to explain the testing approach.

* Restore ability to configure and utilize `updated_at` for snapshots using the check_cols strategy * Changelog entry * Optional comparison of column names starting with `dbt_` * Functional test for check cols snapshots using `updated_at` * Comments to explain the test implementation (cherry picked from commit d09459c)

…) (#5126) * Restore ability to configure and utilize `updated_at` for snapshots using the check_cols strategy * Changelog entry * Optional comparison of column names starting with `dbt_` * Functional test for check cols snapshots using `updated_at` * Comments to explain the test implementation (cherry picked from commit d09459c) Co-authored-by: Doug Beatty <[email protected]>

…-labs#5077) * Restore ability to configure and utilize `updated_at` for snapshots using the check_cols strategy * Changelog entry * Optional comparison of column names starting with `dbt_` * Functional test for check cols snapshots using `updated_at` * Comments to explain the test implementation

Restore ability to configure and utilize updated_at for snapshots u…

3c25db7

…sing the check_cols strategy

cla-bot bot added the cla:yes label Apr 15, 2022

Changelog entry

a513314

jtcohen6 added the Team:Execution label Apr 19, 2022

dbeatty10 added 2 commits April 20, 2022 12:43

Optional comparison of column names starting with dbt_

f4af9f6

Functional test for check cols snapshots using updated_at

6301a10

dbeatty10 marked this pull request as ready for review April 20, 2022 19:03

dbeatty10 requested a review from a team April 20, 2022 19:03

dbeatty10 requested review from a team as code owners April 20, 2022 19:03

dbeatty10 requested review from iknox-fa and McKnight-42 April 20, 2022 19:03

stu-k approved these changes Apr 20, 2022

View reviewed changes

stu-k requested a review from a team April 20, 2022 19:12

ChenyuLInx approved these changes Apr 20, 2022

View reviewed changes

Comments to explain the test implementation

dfc0d65

ChenyuLInx merged commit d09459c into main Apr 21, 2022

ChenyuLInx deleted the dbeatty/fix-check-snapshot branch April 21, 2022 12:56

ChenyuLInx added the backport 1.1.latest label Apr 21, 2022

jtcohen6 mentioned this pull request Sep 16, 2022

[CT-1196] [Bug] Inconsistency between dbt_valid_from and dbt_valid_to #5867

Closed

2 tasks

aranke mentioned this pull request May 20, 2024

[Backport 1.1.latest] Fix #9907: Add retry to tox to reduce flaky tests due to network failures #10179

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restore ability to utilize `updated_at` for check_cols snapshots #5077

Restore ability to utilize `updated_at` for check_cols snapshots #5077

dbeatty10 commented Apr 15, 2022 •

edited

Loading

cla-bot bot commented Apr 15, 2022

github-actions bot commented Apr 15, 2022

jtcohen6 commented Apr 15, 2022

cla-bot bot commented Apr 15, 2022

ChenyuLInx commented Apr 15, 2022

dbeatty10 commented Apr 15, 2022

stu-k left a comment

ChenyuLInx left a comment

ChenyuLInx Apr 20, 2022

dbeatty10 Apr 20, 2022 •

edited

Loading

dbeatty10 Apr 21, 2022

		}


		def test_simple_snapshot(project):

Restore ability to utilize updated_at for check_cols snapshots #5077

Restore ability to utilize updated_at for check_cols snapshots #5077

Conversation

dbeatty10 commented Apr 15, 2022 • edited Loading

Description

Details

Testing approach

Pros

Cons

Future ideas to consider

Checklist

cla-bot bot commented Apr 15, 2022

github-actions bot commented Apr 15, 2022

jtcohen6 commented Apr 15, 2022

cla-bot bot commented Apr 15, 2022

ChenyuLInx commented Apr 15, 2022

dbeatty10 commented Apr 15, 2022

stu-k left a comment

Choose a reason for hiding this comment

ChenyuLInx left a comment

Choose a reason for hiding this comment

ChenyuLInx Apr 20, 2022

Choose a reason for hiding this comment

dbeatty10 Apr 20, 2022 • edited Loading

Choose a reason for hiding this comment

dbeatty10 Apr 21, 2022

Choose a reason for hiding this comment

Restore ability to utilize `updated_at` for check_cols snapshots #5077

Restore ability to utilize `updated_at` for check_cols snapshots #5077

dbeatty10 commented Apr 15, 2022 •

edited

Loading

dbeatty10 Apr 20, 2022 •

edited

Loading