Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Requirement for nullable test data #48

Open
glatterf42 opened this issue Feb 12, 2024 · 0 comments
Open

Requirement for nullable test data #48

glatterf42 opened this issue Feb 12, 2024 · 0 comments
Labels
question Further information is requested

Comments

@glatterf42
Copy link
Member

Not sure this is an actual issue, this needs some clarification first.

As part of #41, I tried updating the pandas version to 2.2.0. This showed numerous DeprecationWarnings and FutureWarnings, one of which was related to .fillna(np.nan) and can be found in pandas release notes. My solution was to simply remove the fillna(np.nan) part of the offending line since this did not affect our results. As @meksor pointed out, we should however check if this is a requirement we want to keep and clarify what exactly this requirement is. Does this only work because our test data only contain np.nan whenever a data cell in a table would be empty? Is there some implicit conversion going on in some background step? For example, pandas.read_sql() has a parameter called dtype_backend, which defaults to "numpy_nullable". Does this convert missing values to np.nan?

Possibly related: a migration guide to copy-on-write behavior.

@glatterf42 glatterf42 added the question Further information is requested label Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant