Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: converting floats to decimals before schema validation #2041

Closed
wants to merge 1 commit into from

Conversation

taylorbarstow
Copy link

@taylorbarstow taylorbarstow commented Nov 12, 2023

Fixes a known challenge with certain validations of floating points using JSON schema, by converting floats to decimals prior to validation.

For example, this fixes errors like:

ValidationError: 92.6 is not a multiple of 0.1

There is a relevant discussion here: https://gitlab.com/meltano/target-csv/-/issues/3

Certain targets (for example: gupy-io/target-s3-parquet) rely on this project to do schema validation, so by handling this here, developers can continue to use that pattern while also avoiding these challenges with floating point validation.


📚 Documentation preview 📚: https://meltano-sdk--2041.org.readthedocs.build/en/2041/

Copy link

codspeed-hq bot commented Nov 12, 2023

CodSpeed Performance Report

Merging #2041 will not alter performance

Comparing SaaSWorksInc:float-to-decimal (a0ab811) with main (466d60a)

Summary

✅ 1 untouched benchmarks

Copy link

codecov bot commented Nov 12, 2023

Codecov Report

Merging #2041 (a0ab811) into main (466d60a) will decrease coverage by 0.02%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##             main    #2041      +/-   ##
==========================================
- Coverage   86.83%   86.82%   -0.02%     
==========================================
  Files          58       58              
  Lines        4885     4894       +9     
  Branches      777      782       +5     
==========================================
+ Hits         4242     4249       +7     
- Misses        456      457       +1     
- Partials      187      188       +1     
Files Coverage Δ
singer_sdk/sinks/core.py 93.33% <100.00%> (ø)
singer_sdk/helpers/_typing.py 74.20% <77.77%> (+0.15%) ⬆️

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

@edgarrmondragon
Copy link
Collaborator

This seems to be intended to fix https://github.com/gupy-io/target-s3-parquet.

Repeating here my comment from Slack:

That target seems to be using an old version of the SDK: https://github.com/gupy-io/target-s3-parquet/blob/a8022b4eef8cf0e803d579a6e8c464d5f5df9b31/pyproject.toml#L15.

My guess is that simply bumping the dependency will fix this issue, since this was addressed in #1809 and shipped with v0.29.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants