dbt 0.20.0rc1 not finding test on model itself

[joshpeng]

Describe the bug

I am using dbt_utils 0.7.0's equality test like this:

version: 2

models:
  - name: model_name
    tests:
      - dbt_utils.equality:
          compare_model: ref('other_table_name')

When I run dbt test -m model_name it says WARNING: Nothing to do. as if it couldn't find any tests to run. If I do dbt test or dbt test -m model_name+ it does find and runs the test.

If I downgrade to dbt 0.19.0 and dbt_utils 0.6.6, the same test is found at dbt test -m model_name and not as a downstream test of the model.

Expected behavior

Believe the test should have been detected on the main model node of -m model_name and not as a downstream test like it did in dbt 0.19.0.

System information

Which database are you using dbt with?

• postgres
• redshift
• bigquery
• snowflake
• other (specify: ____________)

The output of dbt --version:

0.20.0-rc1

The operating system you're using:
Mac

The output of python --version:
3.8

[jtcohen6]

Hey @joshpeng! This is intended as a feature in v0.20.0, implemented in #3235. (Original issue: #2891)

Test selection has always done something pretty subtle, which is the jump from "select a model" to "select all tests that directly depend on that model." We decided that dbt test -m model_name should be be a bit less greedy: it should select tests that depend model_name and only model_name, and which do not depend on any other resource—unless that other resource is also selected.

The equality test in your example depends on both model_name and other_table_name. Previously, dbt test -m model_name would fail if other_table_name did not yet exist in your schema. Starting in dbt v0.20, dbt test -m model_name will no longer indirectly select the equality test, because it depends on an unselected resource. In the following example, dbt test -m model_name would still select the unique test:

version: 2

models:
  - name: model_name
    tests:
      - dbt_utils.equality:
          compare_model: ref('other_table_name')
    columns:
      - name: id
        tests:
          - unique

All of the following selection criteria will select both the unique + equality tests:

dbt test -m model_name other_table_name  # both parents selected
dbt test -m model_name+1
dbt test -m other_table_name+1
dbt test -m @model_name

In the first case, the test is selected indirectly because both of its parents are selected. In the other three cases, the test is selected directly by a graph selection method.

There are prerelease docs for this new behavior here: https://next.docs.getdbt.com/reference/node-selection/test-selection-examples

The final piece of context here is that we're working toward a dbt build command (run + test + seed + snapshot) for v0.21 (#2743). We wanted to make sure that, starting from a totally fresh schema, each of the following would succeed, and have results that make sense:

dbt build -s model_name
dbt build -s other_table_name
dbt build -s model_name other_table_name  # run both models, unique test, equality test

Let's say the definition of other_table_name is just select * from {{ ref('model_name') }}. Then those would have the following behavior:

1. Run model_name + unique test
2. Run other_table_name
3. Run model_name, unique test, other_table_name, equality test (in that order, skipping downstream if anything fails)

I'm going to close this issue for the time being, but I'm still curious to hear your thoughts: Does this make sense to you? Do those docs help in clarifying the behavior?