Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

S3 source: Legacy parquet configs should set decimal_as_float to True #29227

Closed
1 task
girarda opened this issue Aug 8, 2023 · 0 comments · Fixed by #29342
Closed
1 task

S3 source: Legacy parquet configs should set decimal_as_float to True #29227

girarda opened this issue Aug 8, 2023 · 0 comments · Fixed by #29342
Assignees
@girarda
Labels
team/extensibility

Comments

@girarda
Copy link
Contributor

girarda commented Aug 8, 2023

What

The CDK parquet parser converts decimal fields to string by default to ensure there is no loss in precision.

The legacy S3 source returns decimal fields as numbers.

Legacy parquet configs should be adapted to set decimal_as_float to True

Acceptance criteria

  • Legacy S3 avro configs set decimal_as_float to True
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@girarda girarda

Labels
team/extensibility
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

✨Source S3 (v4): Set decimal_as_float to True for parquet files airbytehq/airbyte
1 participant
@girarda