Skip to content

Commit

Permalink
cast high to float in jitter (#1864)
Browse files Browse the repository at this point in the history
<!--Please ensure the PR fulfills the following requirements! -->
<!-- If this is your first PR, make sure to add your details to the
AUTHORS.rst! -->
### Pull Request Checklist:
- [ ] This PR addresses an already opened issue (for bug fixes /
features)
    - This PR fixes #xyz
- [ ] Tests for the changes have been added (for bug fixes / features)
- [ ] (If applicable) Documentation has been added / updated (for bug
fixes / features)
- [x] CHANGELOG.rst has been updated (with summary of main changes)
- [x] Link to issue (:issue:`number`) and pull request (:pull:`number`)
has been added

### What kind of change does this PR introduce?

* I am casting `high` to be a float, not an array. In the new version of
dask, if `high` is an array, there are memory problems and we are unable
to run jitter_under on the ESPO regions.
* I see that it was made explictly an array in #1721. @Zeitsperre do you
remember why ?

### Does this PR introduce a breaking change?
no

### Other information:
This is only a problem when using dask.
  • Loading branch information
juliettelavoie authored Aug 5, 2024
2 parents 4226f57 + ac342ce commit f475729
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 8 deletions.
3 changes: 2 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Changelog

v0.52.0 (unreleased)
--------------------
Contributors to this version: David Huard (:user:`huard`), Trevor James Smith (:user:`Zeitsperre`), Hui-Min Wang (:user:`Hem-W`), Éric Dupuis (:user:`coxipi`), Sarah Gammon (:user:`SarahG-579462`), Pascal Bourgault (:user:`aulemahal`).
Contributors to this version: David Huard (:user:`huard`), Trevor James Smith (:user:`Zeitsperre`), Hui-Min Wang (:user:`Hem-W`), Éric Dupuis (:user:`coxipi`), Sarah Gammon (:user:`SarahG-579462`), Pascal Bourgault (:user:`aulemahal`), Juliette Lavoie (:user:`juliettelavoie`).

New features and enhancements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand All @@ -26,6 +26,7 @@ Bug fixes
* Fixed the indexer bug in the ``xclim.indices.standardized_index_fit_params`` when multiple or non-array indexers are specified and fitted parameters are reloaded from netCDF. (:issue:`1842`, :pull:`1843`).
* Addressed a bug found in ``wet_spell_*`` indicators that was contributing to erroneous results. A new generic spell length statistic function ``xclim.indices.generic.spell_length_statistics`` is now used in wet and dry spells indicators. (:issue:`1834`, :pull:`1838`).
* Syntax for ``nan`` and ``inf`` was adapted to support ``numpy>=2.0.0``. (:pull:`1814`, :issue:`1785`).
* Force type in `jitter` to work with new version of dask. (:pull:`1864`)

Internal changes
^^^^^^^^^^^^^^^^
Expand Down
6 changes: 6 additions & 0 deletions tests/test_indices.py
Original file line number Diff line number Diff line change
Expand Up @@ -610,6 +610,12 @@ class TestStandardizedIndices:
def test_standardized_precipitation_index(
self, open_dataset, freq, window, dist, method, values, diff_tol
):
if (
method == "ML"
and freq == "D"
and Version(__numpy_version__) < Version("2.0.0")
):
pytest.skip("Skipping SPI/ML/D on older numpy")
ds = open_dataset("sdba/CanESM2_1950-2100.nc").isel(location=1)
if freq == "D":
ds = ds.convert_calendar("366_day", missing=np.nan)
Expand Down
20 changes: 13 additions & 7 deletions xclim/sdba/processing.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,14 +218,17 @@ def jitter(
out: xr.DataArray = x
notnull = x.notnull()
if lower is not None:
jitter_lower = np.array(convert_units_to(lower, x)).astype(float)
jitter_min = np.array(
jitter_lower = float(convert_units_to(lower, x))
jitter_min = float(
convert_units_to(minimum, x) if minimum is not None else 0
).astype(float)
)
jitter_min = jitter_min + np.finfo(x.dtype).eps
if uses_dask(x):
jitter_dist = dsk.random.uniform(
low=jitter_min, high=jitter_lower, size=x.shape, chunks=x.chunks
low=jitter_min,
high=jitter_lower,
size=x.shape,
chunks=x.chunks,
)
else:
jitter_dist = np.random.uniform(
Expand All @@ -237,11 +240,14 @@ def jitter(
if upper is not None:
if maximum is None:
raise ValueError("If 'upper' is given, so must 'maximum'.")
jitter_upper = np.array(convert_units_to(upper, x)).astype(float)
jitter_max = np.array(convert_units_to(maximum, x)).astype(float)
jitter_upper = float(convert_units_to(upper, x))
jitter_max = float(convert_units_to(maximum, x))
if uses_dask(x):
jitter_dist = dsk.random.uniform(
low=jitter_upper, high=jitter_max, size=x.shape, chunks=x.chunks
low=jitter_upper,
high=jitter_max,
size=x.shape,
chunks=x.chunks,
)
else:
jitter_dist = np.random.uniform(
Expand Down

0 comments on commit f475729

Please sign in to comment.