Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snowflake: get_tables_by_pattern_sql handling external tables #351

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ If you were relying on the position to match up your optional arguments, this ma

## Fixes
* Handle booleans gracefully in the unpivot macro ([#305](https://github.com/fishtown-analytics/dbt-utils/pull/305) [@avishalom](https://github.com/avishalom))
* Fix a bug in `get_relation_by_prefix` that happens with Snowflake external tables. Now the macro will retrieve tables that match the prefix which are external tables ([#350](https://github.com/fishtown-analytics/dbt-utils/issues/350))


# dbt-utils v0.6.4
Expand Down
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1088,9 +1088,10 @@ We welcome contributions to this repo! To contribute a new feature or a fix, ple

### Dispatch macros

**Note:** This is primarily relevant to users and maintainers of community-supported
database plugins. If you use Postgres, Redshift, Snowflake, or Bigquery, this likely
does not apply to you.
**Note:** This is primarily relevant to:
- Users and maintainers of community-supported [adapter plugins](https://docs.getdbt.com/docs/available-adapters)
- Users who wish to override a low-lying `dbt_utils` macro with a custom implementation, and have that implementation used by other `dbt_utils` macros
If you use Postgres, Redshift, Snowflake, or Bigquery, this likely does not apply to you.

dbt v0.18.0 introduces `adapter.dispatch()`, a reliable way to define different implementations of the same macro
across different databases.
Expand All @@ -1101,15 +1102,17 @@ variable in your project, when dbt searches for implementations of a dispatched
`dbt_utils` macro, it will search through your listed packages _before_ using
the implementations defined in `dbt_utils`.

Set the variable:
Set a variable in your `dbt_project.yml`:
```yml
vars:
dbt_utils_dispatch_list:
- first_package_to_search # likely the name of your root project
- first_package_to_search # likely the name of your root project (only the root folder)
- second_package_to_search # likely an "add-on" package, such as spark_utils
# dbt_utils is always the last place searched
```

If overriding a dispatched macro with a custom implementation in your own project's `macros/` directory, you must name your custom macro with a prefix: either `default__` (note the two underscores), or the name of your adapter followed by two underscores. For example, if you're running on Postgres and wish to override the behavior of `dbt_utils.datediff` (such that `dbt_utils.date_spine` will use your version instead), you can do this by defining a macro called either `default__datediff` or `postgres__datediff`.

When running on Spark, if dbt needs to dispatch `dbt_utils.datediff`, it will search for the following in order:
```
first_package_to_search.spark__datediff
Expand Down
2 changes: 2 additions & 0 deletions macros/sql/get_tables_by_pattern_sql.sql
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
table_name as "table_name",
case table_type
when 'BASE TABLE' then 'table'
when 'EXTERNAL TABLE' then 'external'
upjohnc marked this conversation as resolved.
Show resolved Hide resolved
when 'MATERIALIZED VIEW' then 'materializedview'
else lower(table_type)
end as "table_type"
from {{ database }}.information_schema.tables
Expand Down