Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] dbt docs generate does not include Data Type (data stored in iceberg format) #988

Open
2 tasks done
troylepnc opened this issue Feb 21, 2024 · 2 comments
Open
2 tasks done
Labels
bug Something isn't working

Comments

@troylepnc
Copy link

troylepnc commented Feb 21, 2024

Is this a new bug in dbt-spark?

  • I believe this is a new bug in dbt-spark
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

"dbt docs generate" does not pick up Data Type; column Type is empty

image

Expected Behavior

Column Type shows data type information as specified in dbt model properties (yaml)

Steps To Reproduce

Dbt command: dbt docs generate

models:

  • name: my_test_model
    description: 'my test model'
    meta:
    owner: acme
    columns:
    • name: id
      data_type: bigint
      description: 'Internally managed primary key'
      constraints:
      • type: not_null
    • name: loadcommandid
      data_type: bigint
      description: 'Parent.'
      constraints:
      • type: not_null
    • name: parametername
      data_type: string
      description: 'Parameter name.'
      constraints:
      • type: not_null
    • name: parametervalue
      data_type: string
      description: 'Parameter value.'
    • name: publicid
      data_type: string
      description: 'ID or primary key of the row'
      constraints:
      • type: not_null

Relevant log output

No response

Environment

- OS: Windows 11
- Python: 3.12.1
- Spark: spark-3.5.0-bin-hadoop3-scala2.13
- dbt-core: 1.7.7
- dbt-spark: 1.7.1
- apache iceberg: 1.4.3

Additional Context

"dbt run" and "dbt docs generate" complete successfully with no errors, but data type (column Type) is showing blank in docs output

@troylepnc troylepnc added bug Something isn't working triage labels Feb 21, 2024
@troylepnc troylepnc changed the title [Bug] dbt docs generate does not include Data Type [Bug] [iceberg] dbt docs generate does not include Data Type Feb 21, 2024
@troylepnc troylepnc changed the title [Bug] [iceberg] dbt docs generate does not include Data Type [Bug] dbt docs generate does not include Data Type (data stored in iceberg format) Feb 21, 2024
@Fleid Fleid removed the triage label Feb 22, 2024
@nyc-de
Copy link

nyc-de commented Jun 20, 2024

I ran into this as well. I randomly tried replacing the data_type key with type and it worked.

In the screenshot you can see the docs generate is picking up the column type where the key is replaced.

Screenshot 2024-06-20 at 2 18 11 PM Screenshot 2024-06-20 at 2 18 25 PM

@mabubakr007
Copy link

I ran into this as well. I randomly tried replacing the data_type key with type and it worked.

In the screenshot you can see the docs generate is picking up the column type where the key is replaced.

Screenshot 2024-06-20 at 2 18 11 PM Screenshot 2024-06-20 at 2 18 25 PM

Thanks for this. It worked for us

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants