Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output '*' in union_relations when no columns are found in compile mode #832

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

martinshjung
Copy link

@martinshjung martinshjung commented Sep 7, 2023

This is to handle the scenario with the union_relations macro where the upstream relations haven't been run, and you run SQLFluff against a model using it, with source_column_name=None. Currently the macro outputs nothing after the SELECT statement, resulting in parsing errors with SQLFluff.

Resolves #831.

This is a:

  • documentation update
  • bug fix with no breaking changes
  • new functionality
  • a breaking change

All pull requests from community contributors should target the main branch (default).

Description & motivation

As described in #831, I want to be able to run SQLFluff against models that use union_relation, where their upstream dbt models haven't been built yet, without getting a parser error. This is useful in development/CI workflows, where you want to run linting against newly created models without building them first.

This is essentially the same fix as with the star macro.

Checklist

  • This code is associated with an Issue which has been triaged and accepted for development.
  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake
  • I followed guidelines to ensure that my changes will work on "non-core" adapters by:
    • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
    • using the limit_zero() macro in place of the literal string: limit 0
    • using dbt.type_* macros instead of explicit datatypes (e.g. dbt.type_timestamp() instead of TIMESTAMP
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

@martinshjung
Copy link
Author

martinshjung commented Sep 7, 2023

Happy to receive guidance on how to write a test for this if needed - not sure what the best way is because I have to hook into flags.WHICH == compile, and introspect the output SQL. I've manually run SQLFluff and verified this fixes my issue.

Copy link

github-actions bot commented Mar 6, 2024

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

@github-actions github-actions bot added the Stale label Mar 6, 2024
Copy link

Although we are closing this PR as stale, it can still be reopened to continue development. Just add a comment to notify the maintainers.

@github-actions github-actions bot closed this Mar 13, 2024
@karenderer
Copy link

Hi I encountered the bug described in #831 too - would it be possible to get this reopened/merged?

@dbeatty10
Copy link
Contributor

Thanks for reaching out @karenderer -- I'll re-open this PR and the associated issue.

@dbeatty10 dbeatty10 reopened this Jun 27, 2024
@github-actions github-actions bot removed the Stale label Jun 28, 2024
This is to handle the scenario where the upstream relations haven't
been run, and you run SQLFluff against a model using union_relations,
with source_column_name=None.
@martinshjung martinshjung force-pushed the union-relations-non-existent-upstream-relations branch from 7819b9d to fcb081e Compare July 2, 2024 03:01
@martinshjung martinshjung force-pushed the union-relations-non-existent-upstream-relations branch from fcb081e to 47ac8c0 Compare July 2, 2024 03:02
@martinshjung
Copy link
Author

@dbeatty10 @karenderer I've resolved the merge conflict, so this should be ready for review again.

@benedikt-buchert
Copy link

benedikt-buchert commented Oct 29, 2024

Hi, what is needed to get this merged. Currently, using the code from the patch as a workaround in BigQuery works perfectly! Using this to get my pre-commit in ci working again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Linting union_relation with SQLFluff gives an empty select
4 participants