Better handle returned dask npartitions in tests #1107
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a workaround for issue #1106.
When we request
npartitions=2
when converting apandas
to adask
DataFrame
, we may receive 1 or 2 partitions depending on the version ofdask
. Line and area tests usingaxis=0
have different results depending on the number of partitions asdatashader
does not cope withnpartitions > 1
for theseaxis=0
situations.The workaround here is to check the number of partitions in the returned
dask.DataFrame
to do an appropriate comparison test so that our CI passes regardless ofdask
version. Our functionality is no more broken than it was before, but users are more likely to be seeing inappropriate results withdask >= 2022.8
.