Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle invalid numeric values in JDBC source #1588

Merged
merged 14 commits into from
Jan 13, 2021
Merged

Conversation

sherifnada
Copy link
Contributor

@sherifnada sherifnada commented Jan 8, 2021

What

NaN and infinity are not supported in JSON schema, but are allowed values in various Postgres numeric types . So we force them into null at the DB level.

Releasing this now to fix a bug that a user is having. But we'll want to have tests like this for every data type's edge cases. This is coming in a later PR.

@michel-tricot
Copy link
Contributor

I am afraid that this is adding transformation logic to the source. What would be another option?

@sherifnada sherifnada requested review from cgardens and jrhizor and removed request for jrhizor, michel-tricot and cgardens January 13, 2021 19:29
Copy link
Contributor

@jrhizor jrhizor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unfortunate that NaN/null/infinity will look the same on the destination, but I can't think of a better way right now.

This behavior should be added to our data types documentation.

@sherifnada sherifnada merged commit 68ecf99 into master Jan 13, 2021
@sherifnada sherifnada deleted the sherif/chickenscratch branch January 13, 2021 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants