Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for a generate_alias_name macro. #1350

Closed
nydnarb opened this issue Mar 12, 2019 · 4 comments
Closed

Support for a generate_alias_name macro. #1350

nydnarb opened this issue Mar 12, 2019 · 4 comments

Comments

@nydnarb
Copy link
Contributor

nydnarb commented Mar 12, 2019

Feature

Feature description

dbt users can currently define a generate_schema_name macro. For example, we have a macro

{% macro generate_schema_name(custom_schema_name=none) -%}
    {%- set default_schema = target.schema -%}
    {%- if target.name == 'multi_schema' and custom_schema_name is not none -%}
        {{ custom_schema_name | trim }}
    {%- else -%}
        {{ default_schema }}
    {%- endif -%}
{%- endmacro %}

When the target is set to multi_schema, this macro ensures that the schema defined in the model is used instead of target's default schema. It would be amazing if we could also implement a generate_table_name macro. In many ways, a generate_table_name macro is a more flexible extension of the alias concept that you introduced less than a year ago.

I think this custom macro would be called here: https://github.com/fishtown-analytics/dbt/blob/f6402d3390f4baa7a6f76e546ed3565c79430c60/core/dbt/parser/base.py#L176

Who will this benefit?

Here is my use case. Suppose I have two users, user1 and user2. Each has their own development schema, dbt_user1 and dbt_user2. Now, I have a project that builds two tables foo.baz and bar.baz. foo.baz comes from a SQL file foo__baz.sql with schema='foo' and alias='baz'. The other table bar.baz comes from a file called bar__baz.sql with schema='bar' and alias='baz'. In production, a superuser runs the job and everything is fine.

However, when user1 tries to run the job, they get an error because dbt wants to build dbt_user1.baz twice. If I could write a custom generate_table_name macro, I might make it so that the tables are built as follows in development targets: dbt_user1.foo__baz and dbt_user1.bar__baz.

Note, if the users had appropriate permissions to create schemata, this would not be a problem. For example, these tables would be built in dbt_user1_foo.baz and dbt_user1_bar.baz, as per dbt's documentation on custom schemas. However, we do not have that luxury right now. And I think a generate_table_name macro is, in many ways, the next evolution of alias.

@drewbanin
Copy link
Contributor

Hey @nydnarb - thanks for opening this issue! I think that we're actually pretty well suited to implement something like this. I imagine generate_table_name would have a similar signature to generate_schema_name, eg:

{% macro generate_table_name(model_name, custom_alias=none) -%}

The default implementation would use the custom_alias if it was provided, otherwise it would fall back to the model_name. Cool!

We had a pretty significant problem with the generate_schema_name macro recently, and I think it's something we'll need to consider here too: What happens if users incorporate packages into their project? Should dbt use the generate_table_name defined in the "root" project for all models? Or should it use the implementation specified in the package that contains the model? My instinct is the former, both to retain parity with generate_schema_name, and also because it makes this a lot easier to implement :)

My only other thought is that we'll probably want to call this generate_model_alias or similar, as we try to avoid using the label "table" to refer to things that can be either tables or views.

Is this something you'd be able to help out with implementing? I think the code block you pointed out is definitely the right place for this. We'd just need to implement a function like get_alias_func and provide a default macro implementation in the global project.

@nydnarb
Copy link
Contributor Author

nydnarb commented Mar 12, 2019

I might be able to find some time to contribute. But I might have questions along the way.

@drewbanin
Copy link
Contributor

Cool! Let me know how I can be helpful -- happy to point you in the right direction as you get underway

@nydnarb nydnarb changed the title Support for a generate_table_name macro. Support for a generate_alias_name macro. Mar 20, 2019
@drewbanin drewbanin added this to the Wilt Chamberlain milestone May 1, 2019
@beckjake
Copy link
Contributor

beckjake commented May 2, 2019

Fixed in #1363

@beckjake beckjake closed this as completed May 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants