Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Cluster order for Materialized Views is mixed up #1116

Open
2 tasks done
dennishendricks opened this issue Feb 22, 2024 · 4 comments · May be fixed by #1259
Open
2 tasks done

[Bug] Cluster order for Materialized Views is mixed up #1116

dennishendricks opened this issue Feb 22, 2024 · 4 comments · May be fixed by #1259
Labels
bug Something isn't working materialized_views

Comments

@dennishendricks
Copy link

Is this a new bug in dbt-bigquery?

  • I believe this is a new bug in dbt-bigquery
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

After setting clusters for a materialized view via cluster_by in the Config macro, the order of the clusters is not adopted in the appropriate order as with other table forms, but is mixed according to a pattern that is not apparent.

Expected Behavior

Cluster order is adopted 1:1.

Steps To Reproduce

  1. Create a model as "materialized_view"
  2. Set clusters via cluster_by parameter
  3. See mixed cluster order in target -> run folder or result in BigQuery console

Example model:

{{
    config(
        materialized="materialized_view",
        cluster_by=[ "portal", "cu_name", "rt_name", "ids_in_container"],
    )
}}

select * from {{ ref('orders') }}

Result in target folder:

create materialized view if not exists `xxx`
    cluster by rt_name, ids_in_container, portal, cu_name

Relevant log output

No response

Environment

- OS: MacOS Sonoma 14.0
- Python: 3.11.1
- dbt-core: 1.7.8
- dbt-bigquery: 1.7.5

Additional Context

No response

@dennishendricks dennishendricks added bug Something isn't working triage labels Feb 22, 2024
@dbeatty10
Copy link
Contributor

Thanks for reporting this @dennishendricks !

I didn't confirm or deny, but I suspect this is involved:

Since the cluster key is being converted into an unordered set rather than an ordered list, then the keys aren't guaranteed to preserve the order by the time we reach here.

@dbeatty10 dbeatty10 removed the triage label Feb 22, 2024
@thmswt thmswt linked a pull request Jun 13, 2024 that will close this issue
4 tasks
@thmswt
Copy link

thmswt commented Jun 19, 2024

Hi @dbeatty10, I have created a PR that fixes this bug. Would you mind taking a look at it?
Thanks in advance.

@mikealfare
Copy link
Contributor

I added the triage label to get this back on the Product backlog given the new PR.

@thmswt
Copy link

thmswt commented Jul 16, 2024

Hi @mikealfare, I just updated the PR to the latest changes from main. The PR is ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working materialized_views
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants