Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In projects named warehouse, Snowflake warehouse config gets confused #1899

Closed
1 task done
jtcohen6 opened this issue Nov 7, 2019 · 3 comments
Closed
1 task done
Labels
bug Something isn't working snowflake

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Nov 7, 2019

Describe the bug

If a dbt project is named warehouse, almost all Snowflake models running in the 0.15.0 beta release experience the following error:

Database Error in model my_model (models/my_model.sql)
  001003 (42000): SQL compilation error:
  syntax error line 1 at position 14 unexpected '{'.
  compiled SQL at target/compiled/my_project/my_model.sql

This is caused by the changes in #1788, which allow per-model Snowflake warehouse specification using a warehouse config argument. If a dbt project is also named warehouse, confusion ensues.

In an example dbt_project.yml:

name: warehouse
. . .
models:
  warehouse:
    enabled: true
    materialized: view
    schema: analytics

For all models that do not have a warehouse config argument specified, Instead of skipping the use warehouse statement altogether, dbt is running

On my_model: use warehouse {'enabled': True, 'materialized': 'view', 'schema': 'analytics'}

I believe the relevant line is here of pre_model_hook, whereby config.get('warehouse', default_warehouse) returns... not the warehouse we're looking for.

Steps To Reproduce

Name your dbt project warehouse, and run any model on Snowflake without a warehouse config.

Expected behavior

A model with no warehouse config should always behave exactly the same as a model configured

{{config(
    warehouse = target.warehouse
) }}

System information

Which database are you using dbt with?

  • snowflake

The output of dbt --version:

installed version: 0.15.0-b2
   latest version: 0.14.3

Your version of dbt is ahead of the latest release!

Additional context

The easiest solution here is to rename the dbt project to something other than warehouse. For that reason, I would understand if this is a #wontfix

@jtcohen6 jtcohen6 added bug Something isn't working triage labels Nov 7, 2019
@beckjake
Copy link
Contributor

beckjake commented Nov 7, 2019

Oh no, this is terrible. I think we need to add warehouse to the list of blacklisted project names in contracts/project.py, at least.

@jtcohen6
Copy link
Contributor Author

jtcohen6 commented Nov 7, 2019

I just tested a project that is not named warehouse but which does have a models subdirectory named warehouse, with config present in dbt_project.yml. That also results in this error.

Is there a way to more tightly scope the {{config.get()}}? Or enforce that its type must be a str?

@drewbanin
Copy link
Contributor

@beckjake let's rename warehouse to snowflake_warehouse to fix this specific problem.

Let's also revisit this question in a future release to figure out how to make dbt's config structure unambiguous. This will all break if people have projects or paths named sort, bind, dist`, etc etc etc which is not what we want here long term

@drewbanin drewbanin added snowflake and removed triage labels Nov 7, 2019
@drewbanin drewbanin added this to the Louisa May Alcott milestone Nov 7, 2019
beckjake added a commit that referenced this issue Nov 8, 2019
…wflake-warehouse

rename "warehouse" config key to "snowflake_warehouse" (#1899)
@beckjake beckjake closed this as completed Nov 8, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working snowflake
Projects
None yet
Development

No branches or pull requests

3 participants