Tests for variables with units #3654

keewis · 2019-12-28T20:21:06Z

As promised in #3493, this adds integration tests for units. I'm doing this now rather than later since I encountered a few cases in #3643 where a increased test coverage for variables would have been helpful.

Tests added
Passes black . && mypy . && flake8

keewis · 2019-12-28T20:29:58Z

xarray/tests/test_units.py

+def delete_attrs(*to_delete):
+    def wrapper(cls):
+        for item in to_delete:
+            setattr(cls, item, None)
+
+        return cls
+
+    return wrapper
+
+
+@delete_attrs(
+    "test_getitem_with_mask",
+    "test_getitem_with_mask_nd_indexer",


are there better ways to exclude inherited tests? I'd like to avoid marking them as skipped since they will probably never pass.

No idea. You could split the parent class into two I guess?

Maybe @max-sixty knows some tricks.

keewis · 2020-01-08T23:32:53Z

this should be ready for review and merge.

Also, I found a few issues that I'm posting here so I won't forget about them:

documentation:
- the examples of Variable.rolling_window pass a variable x that is not set anywhere and would change the results
- the docstring of set_dims is quite confusing (to me, at least), it does not mention the shape parameter (which would be the same as passing a dictionary with values dict(zip(dims, shape)) to dims) or that broadcasting is used to expand if necessary (so no reshaping).
code: pad_with_fill_value seems to implement pad for dask arrays for all versions of dask, but at least for recent versions we should rely dask's implementation instead

I can update the documentation, but I'd leave the revisiting of the dask issues (there are a lot of them) to someone else.

keewis · 2020-01-09T00:06:02Z

the test failures are due to the pint update which means they should not be related. I'll mark them as xfail and fix them in the appropriate PR (#3611 for the apply_ufunc failures and the Dataset PR I will open once all Variable and DataArray tests are passing for the to_stacked_array failures)

keewis · 2020-01-09T00:09:47Z

this should probably be merged before any of the other related PRs

xarray/tests/test_units.py

dcherian · 2020-01-14T01:51:13Z

xarray/tests/test_units.py

@@ -28,6 +29,21 @@
 ]


+def is_compatible(unit1, unit2):


seems like a nice thing to move upstream if pint wants it.

👍, I'll ask

Edit: see hgrecco/pint#988

xarray/tests/test_units.py

dcherian · 2020-01-14T01:56:36Z

xarray/tests/test_units.py

+def delete_attrs(*to_delete):
+    def wrapper(cls):
+        for item in to_delete:
+            setattr(cls, item, None)
+
+        return cls
+
+    return wrapper
+
+
+@delete_attrs(
+    "test_getitem_with_mask",
+    "test_getitem_with_mask_nd_indexer",


No idea. You could split the parent class into two I guess?

Maybe @max-sixty knows some tricks.

xarray/tests/test_units.py

dcherian · 2020-01-14T16:29:00Z

xarray/tests/test_units.py

+
+        assert expected == actual
+
+    @pytest.mark.parametrize(


I stopped here. Will look at the rest tomorrow.

dcherian

just some minor comments. Feel free to merge when you're ready.

Thanks @keewis. It's impressive that so much of this works!

xarray/tests/test_units.py

keewis · 2020-01-15T16:31:07Z

I'll merge once the tests pass.

keewis · 2020-01-15T16:59:00Z

thanks for the review, @dcherian

…e-meta * 'master' of github.com:pydata/xarray: Feature/align in dot (pydata#3699) ENH: enable `H5NetCDFStore` to work with already open h5netcdf.File a… (pydata#3618) One-off isort run (pydata#3705) hardcoded xarray.__all__ (pydata#3703) Bump mypy to v0.761 (pydata#3704) remove DataArray and Dataset constructor deprecations for 0.15 (pydata#3560) Tests for variables with units (pydata#3654)

* upstream/master: (23 commits) Feature/align in dot (pydata#3699) ENH: enable `H5NetCDFStore` to work with already open h5netcdf.File a… (pydata#3618) One-off isort run (pydata#3705) hardcoded xarray.__all__ (pydata#3703) Bump mypy to v0.761 (pydata#3704) remove DataArray and Dataset constructor deprecations for 0.15 (pydata#3560) Tests for variables with units (pydata#3654) Add an example notebook using apply_ufunc to vectorize 1D functions (pydata#3629) Use encoding['dtype'] over data.dtype when possible within CFMaskCoder.encode (pydata#3652) allow passing any iterable to drop when dropping variables (pydata#3693) Typo on DataSet/DataArray.to_dict documentation (pydata#3692) Fix mypy type checking tests failure in ds.merge (pydata#3690) Explicitly convert result of pd.to_datetime to a timezone-naive type (pydata#3688) ds.merge(da) bugfix (pydata#3677) fix docstring for combine_first: returns a Dataset (pydata#3683) Add option to choose mfdataset attributes source. (pydata#3498) How do I add a new variable to dataset. (pydata#3679) Add map_blocks example to whats-new (pydata#3682) Make dask names change when chunking Variables by different amounts. (pydata#3584) raise an error when renaming dimensions to existing names (pydata#3645) ...

* 'master' of github.com:pydata/xarray: (27 commits) bump min deps for 0.15 (pydata#3713) setuptools-scm and isort tweaks (pydata#3720) Allow binned coordinates on 1D plots y-axis. (pydata#3685) apply_ufunc: Add meta kwarg + bump dask to 2.2 (pydata#3660) setuptools-scm and one-liner setup.py (pydata#3714) Feature/align in dot (pydata#3699) ENH: enable `H5NetCDFStore` to work with already open h5netcdf.File a… (pydata#3618) One-off isort run (pydata#3705) hardcoded xarray.__all__ (pydata#3703) Bump mypy to v0.761 (pydata#3704) remove DataArray and Dataset constructor deprecations for 0.15 (pydata#3560) Tests for variables with units (pydata#3654) Add an example notebook using apply_ufunc to vectorize 1D functions (pydata#3629) Use encoding['dtype'] over data.dtype when possible within CFMaskCoder.encode (pydata#3652) allow passing any iterable to drop when dropping variables (pydata#3693) Typo on DataSet/DataArray.to_dict documentation (pydata#3692) Fix mypy type checking tests failure in ds.merge (pydata#3690) Explicitly convert result of pd.to_datetime to a timezone-naive type (pydata#3688) ds.merge(da) bugfix (pydata#3677) fix docstring for combine_first: returns a Dataset (pydata#3683) ...

* master: Add support for CFTimeIndex in get_clean_interp_index (pydata#3631) sel with categorical index (pydata#3670) bump min deps for 0.15 (pydata#3713) setuptools-scm and isort tweaks (pydata#3720) Allow binned coordinates on 1D plots y-axis. (pydata#3685) apply_ufunc: Add meta kwarg + bump dask to 2.2 (pydata#3660) setuptools-scm and one-liner setup.py (pydata#3714) Feature/align in dot (pydata#3699) ENH: enable `H5NetCDFStore` to work with already open h5netcdf.File a… (pydata#3618) One-off isort run (pydata#3705) hardcoded xarray.__all__ (pydata#3703) Bump mypy to v0.761 (pydata#3704) remove DataArray and Dataset constructor deprecations for 0.15 (pydata#3560) Tests for variables with units (pydata#3654) Add an example notebook using apply_ufunc to vectorize 1D functions (pydata#3629) Use encoding['dtype'] over data.dtype when possible within CFMaskCoder.encode (pydata#3652)

keewis added 3 commits December 27, 2019 23:49

add tests for variable by inheriting from VariableSubclassobjects

a342a36

make sure the utility functions work with variables

c27cf4c

add additional tests for aggregation and numpy methods

3a7f2a0

keewis commented Dec 28, 2019

View reviewed changes

keewis added 13 commits December 29, 2019 15:02

don't assume variables have a name attribute

b0b1e2c

properly merge the used arguments

1893329

properly attach to non-xarray objects

4058039

add a test function for searchsorted and item

6099585

add tests for missing value detection methods

a7b0f6a

add tests for comparisons

b9338ae

don't try to check missing value behaviour for int arrays

00169c5

xfail the compatible unit tests

a37fada

use an additional check since identical is not sufficient right now

9081c36

check for compatibility by comparing the dimensionality of units

41cb4d7

add tests for fillna

d2258af

add tests for isel

2f6e6df

Merge branch 'master' into tests-for-variables-with-units

df7ca9e

keewis mentioned this pull request Dec 31, 2019

support for units with pint #3594

Open

18 tasks

keewis added 12 commits December 31, 2019 15:38

add tests for 1d math

69bc684

add tests for broadcast_equals

e81029b

add tests for masking

1302d12

add tests for squeeze

59c9edc

add tests for coarsen, quantile, roll, round and shift

bbb4105

add tests for searchsorted

a11885b

add tests for rolling_window

6dba6bc

add tests for transpose

d27cf80

add tests for stack and unstack

5dffc12

add tests for set_dims

cbb8da3

add tests for concat, copy and no_conflicts

e2faede

add tests for reduce

350214b

keewis added 3 commits January 9, 2020 00:12

add tests for pad_with_fill_value

e34a3aa

all and any have been implemented by pint

48df9c5

remove a unnecessary call to np.array

fa9a80b

mark the unrelated failures as xfail

5771d4d

keewis changed the title ~~WIP: Tests for variables with units~~ Tests for variables with units Jan 13, 2020

keewis requested a review from dcherian January 13, 2020 13:40

dcherian reviewed Jan 14, 2020

View reviewed changes

keewis added 5 commits January 14, 2020 17:40

remove a debug print

6da9f4b

document the need for force_ndarray

a30f787

make is_compatible a little bit more robust

61dff86

clean up is_compatible

f67c29e

add docstrings explaining the use of the method and function classes

b412d86

dcherian approved these changes Jan 15, 2020

View reviewed changes

xarray/tests/test_units.py Show resolved Hide resolved

xarray/tests/test_units.py Outdated Show resolved Hide resolved

xarray/tests/test_units.py Outdated Show resolved Hide resolved

keewis added 2 commits January 15, 2020 17:09

put the unit comparisons into a function

5375c9d

actually squeeze the dimensions separately

a852aef

keewis merged commit 3955c37 into pydata:master Jan 15, 2020

keewis deleted the tests-for-variables-with-units branch January 15, 2020 16:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tests for variables with units #3654

Tests for variables with units #3654

keewis commented Dec 28, 2019

keewis Dec 28, 2019 •

edited

Loading

dcherian Jan 14, 2020

keewis commented Jan 8, 2020 •

edited

Loading

keewis commented Jan 9, 2020

keewis commented Jan 9, 2020

dcherian Jan 14, 2020

keewis Jan 14, 2020 •

edited

Loading

dcherian Jan 14, 2020

dcherian Jan 14, 2020

dcherian left a comment

keewis commented Jan 15, 2020

keewis commented Jan 15, 2020

Tests for variables with units #3654

Tests for variables with units #3654

Conversation

keewis commented Dec 28, 2019

keewis Dec 28, 2019 • edited Loading

Choose a reason for hiding this comment

dcherian Jan 14, 2020

Choose a reason for hiding this comment

keewis commented Jan 8, 2020 • edited Loading

keewis commented Jan 9, 2020

keewis commented Jan 9, 2020

dcherian Jan 14, 2020

Choose a reason for hiding this comment

keewis Jan 14, 2020 • edited Loading

Choose a reason for hiding this comment

dcherian Jan 14, 2020

Choose a reason for hiding this comment

dcherian Jan 14, 2020

Choose a reason for hiding this comment

dcherian left a comment

Choose a reason for hiding this comment

keewis commented Jan 15, 2020

keewis commented Jan 15, 2020

keewis Dec 28, 2019 •

edited

Loading

keewis commented Jan 8, 2020 •

edited

Loading

keewis Jan 14, 2020 •

edited

Loading