Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Destination BigQuery : normalization incorrectly processes arrays #13154

Closed
DoNotPanicUA opened this issue May 24, 2022 · 0 comments · Fixed by #13289
Closed

🐛 Destination BigQuery : normalization incorrectly processes arrays #13154

DoNotPanicUA opened this issue May 24, 2022 · 0 comments · Fixed by #13289

Comments

@DoNotPanicUA
Copy link
Contributor

Current Behavior

The normalization surrounds all incoming values by brackets " and then fails or stores it as surrounded text.

JSON schema: data_type_array_test_catalog.txt

Test data: data_type_array_test_messages.txt

  1. Date fail example:
    The result column contains a normalization error instead of data. The root cause of the error is that value is surrounded by brackets.
    Screenshot from 2022-05-24 23-50-41

  2. Text example:
    The value is double surrounded by brackets which leads to an error when you read this data.
    Screenshot from 2022-05-24 23-52-54

@DoNotPanicUA DoNotPanicUA added type/bug Something isn't working needs-triage labels May 24, 2022
DoNotPanicUA added a commit that referenced this issue May 27, 2022
* Enable basic DAT tests

* adopt basic tests + avoid DateTime issue #13123

* disable array tests due to issue #13154

* format

* DON"T MERGE

* fix emitted_at

Signed-off-by: Sergey Chvalyuk <[email protected]>

* Revert "fix emitted_at"

This reverts commit e752a24.

* Revert "DON"T MERGE"

This reverts commit dc2806b.

* google format

Co-authored-by: Sergey Chvalyuk <[email protected]>
@grubberr grubberr self-assigned this May 27, 2022
jscottpolevault pushed a commit to jscottpolevault/airbyte that referenced this issue Jun 1, 2022
* Enable basic DAT tests

* adopt basic tests + avoid DateTime issue #13123

* disable array tests due to issue airbytehq#13154

* format

* DON"T MERGE

* fix emitted_at

Signed-off-by: Sergey Chvalyuk <[email protected]>

* Revert "fix emitted_at"

This reverts commit e752a24.

* Revert "DON"T MERGE"

This reverts commit dc2806b.

* google format

Co-authored-by: Sergey Chvalyuk <[email protected]>
@DoNotPanicUA DoNotPanicUA self-assigned this Jun 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants