Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger full reparse if changes to widely used built-in macros #3810

Closed
jtcohen6 opened this issue Aug 25, 2021 · 1 comment · Fixed by #3982
Closed

Trigger full reparse if changes to widely used built-in macros #3810

jtcohen6 opened this issue Aug 25, 2021 · 1 comment · Fixed by #3982
Assignees
Labels
bug Something isn't working partial_parsing

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Aug 25, 2021

There is a special category of macro that:

  • dbt uses at parse time, across almost all resources in the project
  • is not explicitly called within project code, such that its dependency is not grabbed during static analysis; or is excluded from depends_on.macros because of being so widely used

Those macros are:

  • ref
  • source
  • config
  • execute (?)
  • generate_schema_name
  • generate_database_name
  • generate_alias_name

The builtins are fairly self-explanatory (though execute is a weird inclusion in those docs).

As for generate_x_name macros: The code that uses these macros is incredibly thorny, pretty old, and I think it has to do with a method named generate_generate_component_name_macro. I don't mind hard-coding that these seven need special treatment, and that any detected changes to them should be grounds for triggering a full re-parse (with logging + tracking to show for it).

Opened as a bug because this currently yields incorrect results :)

@jtcohen6
Copy link
Contributor Author

jtcohen6 commented Sep 2, 2021

any detected changes to them should be grounds for triggering a full re-parse

Edit: any detected changes to these macros, or any macros that these macros depend on (via depends_on.macros)

@jtcohen6 jtcohen6 changed the title Trigger full reparse if changes to generate_x_name macro Trigger full reparse if changes to widely used built-in macros Sep 17, 2021
@gshank gshank self-assigned this Sep 20, 2021
gshank added a commit that referenced this issue Oct 1, 2021
[#3810] Skip partial parsing if certain macros have changed
iknox-fa pushed a commit that referenced this issue Feb 8, 2022
automatic commit by git-black, original commits:
  052a306
iknox-fa pushed a commit that referenced this issue Feb 8, 2022
automatic commit by git-black, original commits:
  052a306
  98c015b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working partial_parsing
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants