Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAP-637] [Feature] Follow-up to nested fields #738 and #673 - Allow array of struct #781

Closed
3 tasks done
Tracked by #7372 ...
b-per opened this issue Jun 21, 2023 · 0 comments · Fixed by #806
Closed
3 tasks done
Tracked by #7372 ...

[ADAP-637] [Feature] Follow-up to nested fields #738 and #673 - Allow array of struct #781

b-per opened this issue Jun 21, 2023 · 0 comments · Fixed by #806
Assignees
Labels
enhancement New feature or request

Comments

@b-per
Copy link
Contributor

b-per commented Jun 21, 2023

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-bigquery functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I have tested #738 with a user but while the initial error has been fixed it doesn't support some of the use cases that the person wants to fulfil.

While we can now define a field as a struct there is no way to define a fields as an array of struct.

For a field that is defined at the top level like:

- name: my_array_struct
  description: |
    My overall description

    **field_1**: Desc for field_1
    **field_2**: Desc for field_2
  data_type: |
    array<
      struct<
        field_1 string,
        field_2 numeric>>

Changing it to the following doesn't work due to contract mismatch as there is no way to mention that the top field is an array

- name: my_array_struct.field_1
  data_type: string
  description: Desc for field_1
- name: my_array_struct.field_2
  data_type: numeric
  description: Desc for field_2

BigQuery allows setting constraints and documenting subfields of arrays though.
In that case, the top field is of the type REPEATED (see below).

image

Describe alternatives you've considered

The alternative is to define the parent field but not the subfields.

In that case, we can't define descriptions, constraints or tests on the subfields though.

Who will this benefit?

BigQuery users leveraging arrays of struct and wanting to set contracts/constraints on those while still being able to document and test them.

Are you interested in contributing this feature?

No response

Anything else?

No response

@b-per b-per added enhancement New feature or request triage labels Jun 21, 2023
@github-actions github-actions bot changed the title [Feature] Follow-up to nested fields #738 and #673 - Allow array of struct [ADAP-637] [Feature] Follow-up to nested fields #738 and #673 - Allow array of struct Jun 21, 2023
@jtcohen6 jtcohen6 removed the triage label Jun 24, 2023
@MichelleArk MichelleArk self-assigned this Jul 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
3 participants