Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional where clause to get_column_values #583

Merged
merged 10 commits into from
May 12, 2022
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
- [@graciegoheen](https://github.com/graciegoheen) (#545)
- [@judahrand](https://github.com/judahrand) (#552)
- [@clausherther](https://github.com/clausherther) (#555)
- [@epapineau](https://github.com/epapineau) (#582)
epapineau marked this conversation as resolved.
Show resolved Hide resolved

# dbt-utils v0.8.4
## Fixes
Expand All @@ -22,6 +23,7 @@
- A macro for deduplicating data, `deduplicate()` ([#335](https://github.com/dbt-labs/dbt-utils/issues/335), [#512](https://github.com/dbt-labs/dbt-utils/pull/512))
- A cross-database implementation of `listagg()` ([#530](https://github.com/dbt-labs/dbt-utils/pull/530))
- A new macro to get the columns in a relation as a list, `get_filtered_columns_in_relation()`. This is similar to the `star()` macro, but creates a Jinja list instead of a comma-separated string. ([#516](https://github.com/dbt-labs/dbt-utils/pull/516))
- Add an optional `where` clause parameter to `get_column_values()` to filter values returned ([#511](https://github.com/dbt-labs/dbt-utils/issues/511), [#583](https://github.com/dbt-labs/dbt-utils/pull/583))
epapineau marked this conversation as resolved.
Show resolved Hide resolved

## Fixes
- `get_column_values()` once more raises an error when the model doesn't exist and there is no default provided ([#531](https://github.com/dbt-labs/dbt-utils/issues/531), [#533](https://github.com/dbt-labs/dbt-utils/pull/533))
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -553,6 +553,7 @@ This macro returns the unique values for a column in a given [relation](https://
**Args:**
- `table` (required): a [Relation](https://docs.getdbt.com/reference/dbt-classes#relation) (a `ref` or `source`) that contains the list of columns you wish to select from
- `column` (required): The name of the column you wish to find the column values of
- `where` (optional, default=`none`): A where clause to filter the column values by.
- `order_by` (optional, default=`'count(*) desc'`): How the results should be ordered. The default is to order by `count(*) desc`, i.e. decreasing frequency. Setting this as `'my_column'` will sort alphabetically, while `'min(created_at)'` will sort by when thevalue was first observed.
- `max_records` (optional, default=`none`): The maximum number of column values you want to return
- `default` (optional, default=`[]`): The results this macro should return if the relation has not yet been created (and therefore has no column values).
Expand Down
12 changes: 12 additions & 0 deletions integration_tests/data/sql/data_get_column_values_where.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
field,condition
a,left
b,right
c,left
d,right
e,left
f,right
g,left
g,right
g,left
g,right
g,left
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
field
a
c
e
g
5 changes: 5 additions & 0 deletions integration_tests/models/sql/schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,11 @@ models:
values:
- '5'

- name: test_get_column_values_where
tests:
- dbt_utils.equality:
compare_model: ref('data_get_column_values_where_expected')

- name: test_get_filtered_columns_in_relation
tests:
- dbt_utils.equality:
Expand Down
6 changes: 6 additions & 0 deletions integration_tests/models/sql/test_get_column_values_where.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{% set column_values = dbt_utils.get_column_values(ref('data_get_column_values_where'), 'field', where="condition = 'left'") %}

-- Create a relation using the values
{% for val in column_values -%}
select {{ dbt_utils.string_literal(val) }} as field {% if not loop.last %}union all{% endif %}
{% endfor %}
11 changes: 8 additions & 3 deletions macros/sql/get_column_values.sql
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{% macro get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none) -%}
{{ return(adapter.dispatch('get_column_values', 'dbt_utils')(table, column, order_by, max_records, default)) }}
{% macro get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none, where=none) -%}
{{ return(adapter.dispatch('get_column_values', 'dbt_utils')(table, column, order_by, max_records, default, where)) }}
{% endmacro %}

{% macro default__get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none) -%}
{% macro default__get_column_values(table, column, order_by='count(*) desc', max_records=none, default=none, where=none) -%}
{#-- Prevent querying of db in parsing mode. This works because this macro does not create any new refs. #}
{%- if not execute -%}
{% set default = [] if not default %}
Expand Down Expand Up @@ -37,6 +37,11 @@
{{ column }} as value

from {{ target_relation }}

{% if where is not none %}
where {{ where }}
{% endif %}

group by {{ column }}
order by {{ order_by }}

Expand Down