[CT-2122] [Bug] project variables are None if dbt.config.get is called within string interpolation

[mederka]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

In a Python model like this:

# my_python_model.py
import pandas as pd

def model(dbt, fal):
    dbt.config(materialized="table") 
    print(f"my var: {dbt.config.get('my_var')}") # Prints "my var: None"
    df: pd.DataFrame = dbt.ref("model")
    return df

It works fine if I call dbt.config.get outside of an interpolated string definition:

# my_python_model.py
import pandas as pd

def model(dbt, fal):
    dbt.config(materialized="table")
    my_var = dbt.config.get("my_var") 
    print(f"my var: {my_var}") # Prints "my var: value_of_my_var"
    df: pd.DataFrame = dbt.ref("model")
    return df

Expected Behavior

dbt.config.get should work inside interpolated strings

# my_python_model.py
import pandas as pd

def model(dbt, fal):
    dbt.config(materialized="table") 
    print(f"my var: {dbt.config.get('my_var')}") # Prints "my var: value_of_my_var"
    df: pd.DataFrame = dbt.ref("model")
    return df

Steps To Reproduce

On a linux machine, a Python model definition like this:

# my_python_model.py
import pandas as pd

def model(dbt, fal):
    dbt.config(materialized="table") 
    print(f"my var: {dbt.config.get('my_var')}") # Prints "my var: None"
    df: pd.DataFrame = dbt.ref("model")
    return df

Will print my var: None.

Relevant log output

No response

Environment

- OS: Ubuntu 20.04
- Python: 3.9
- dbt: 1.4

Which database adapter are you using with dbt?

postgres

Additional Context

Error first reported by a dbt-fal user.

[dbeatty10]

Hey @mederka thanks for reporting this! This is weird 🌵

I was able to reproduce this when doing a similar setup as you described. I tested two different setups:

• dbt-duckdb + dbt-fal
• dbt-duckdb on its own

My setup included setting a var in models/schema.yml like this:

version: 2

models:
  - name: my_python_model
    config:
      my_var: my_special_value
      another_var: bar

Variations that work

These all fully worked:

    my_value = dbt.config.get("my_var")
    print(f"{my_value}")

and oddly this too

    dbt.config.get("my_var")
    print(f"{dbt.config.get('my_var')}")

and even odder with this backwards ordering

    print(f"{dbt.config.get('my_var')}")
    dbt.config.get("my_var")

and this too

    print(f"{dbt.config.get('my_var')}")
    print(f"{dbt.config.get('another_var')}")
    dbt.config.get("my_var")
    dbt.config.get("another_var")

Variations that don't work

But none of these fully worked:

    print(f"{dbt.config.get('my_var')}")

or

    print(f"{dbt.config.get('my_var')}")
    dbt.config.get("another_var")

Next steps

I wouldn't expect this to affect users of dbt-snowflake, dbt-bigquery, or dbt-databricks (that aren't also using dbt-fal) because none of them can print to standard out using Python print() (since the Python is shipped off to the data platform).

It also worked when I tried shoving f"{dbt.config.get('my_var')}" into a data frame in dbt-snowflake. (I only tried it without dbt-fal.)

I'm going to mark this as help_wanted for motivated community member to pick up.

[jtcohen6]

We statically analyze the dbt.config.get calls at parse time, to figure out which configs to supply to the config method of dbtObj at runtime.

My understanding would be, this static analysis works for:

dbt.config.get("my_var")

But not when wrapped inside an f-string:

print(f"{dbt.config.get('my_var')}")

This might be a bit surprising, but it doesn't feel high priority, since there are plenty of ways to work around it. If someone is interested in looking more deeply at how f-strings are represented inside Python's AST...

dbt-core/core/dbt/parser/models.py

Lines 206 to 209 in 21fae1c

	dbt_parser = PythonParseVisitor(node)
	dbt_parser.visit(tree)
	for (func, args, kwargs) in dbt_parser.dbt_function_calls: