Numpy 2 compatibility #3491

dopplershift · 2024-04-26T21:24:44Z

Description Of Changes

Various changes to get things to work without warnings/errors on numpy 2 since I was able to put together a local pip-based numpy 2 environment. Mostly avoiding a DeprecationWarning for using trapz instead of trapezoid, otherwise a bunch of silly stuff. This includes an error caught by Matplotlib 3.9.0rc2, which I'm not at all clear why didn't flag on the nightly build (though it had other image-based failures).

This PR (plus hgrecco/pint#1971) leaves us with 4 failing tests. 2 seemingly due to some xarray+cartopy challenges:

___________________________________________________________________________________________________________ test_declarative_events ___________________________________________________________________________________________________________
args = (), kwargs = {}
def wrapper(*args, **kwargs):


  store.return_value[test_name] = obj(*args, **kwargs)


../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/pytest_mpl/plugin.py:125:

src/metpy/testing.py:122: in wrapped

return test_func(*args, **kwargs)

tests/plots/test_declarative.py:495: in test_declarative_events

pc.draw()

src/metpy/plots/declarative.py:187: in draw

panel.draw()

src/metpy/plots/declarative.py:511: in draw

p.draw()

src/metpy/plots/declarative.py:891: in draw

self._build()

src/metpy/plots/declarative.py:1020: in _build

self.handle = self.parent.ax.imshow(imdata, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/mpl/geoaxes.py:307: in wrapper

return func(self, *args, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/mpl/geoaxes.py:1314: in imshow

img, extent = warp_array(img,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/img_transform.py:182: in warp_array

source_native_xy = mesh_projection(source_proj, nx, ny,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/img_transform.py:77: in mesh_projection

x, xstep = np.linspace(x_lower, x_upper, nx, retstep=True,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/numpy/_core/function_base.py:189: in linspace

y = conv.wrap(y.astype(dtype, copy=False))

../xarray/xarray/core/dataarray.py:4674: in array_wrap

new_var = self.variable.array_wrap(obj, context)

../xarray/xarray/core/variable.py:2290: in array_wrap

return Variable(self.dims, obj)

../xarray/xarray/core/variable.py:397: in init

super().init(

../xarray/xarray/namedarray/core.py:264: in init

self._dims = self._parse_dimensions(dims)

self = <[AttributeError("'Variable' object has no attribute '_dims'") raised in repr()] Variable object at 0x16c0ea740>, dims = ()
def _parse_dimensions(self, dims: _DimsLike) -> _Dims:
    dims = (dims,) if isinstance(dims, str) else tuple(dims)
    if len(dims) != self.ndim:


      raise ValueError(


            f"dimensions {dims} must have the same length as the "
            f"number of data dimensions, ndim={self.ndim}"
        )

E           ValueError: dimensions () must have the same length as the number of data dimensions, ndim=1
../xarray/xarray/namedarray/core.py:490: ValueError

_________________________________________________________________________________________________________________ test_latlon _________________________________________________________________________________________________________________
args = (), kwargs = {}
def wrapper(*args, **kwargs):


  store.return_value[test_name] = obj(*args, **kwargs)


../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/pytest_mpl/plugin.py:125:

src/metpy/testing.py:122: in wrapped

return test_func(*args, **kwargs)

tests/plots/test_declarative.py:838: in test_latlon

pc.draw()

src/metpy/plots/declarative.py:187: in draw

panel.draw()

src/metpy/plots/declarative.py:511: in draw

p.draw()

src/metpy/plots/declarative.py:891: in draw

self._build()

src/metpy/plots/declarative.py:1020: in _build

self.handle = self.parent.ax.imshow(imdata, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/mpl/geoaxes.py:307: in wrapper

return func(self, *args, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/mpl/geoaxes.py:1314: in imshow

img, extent = warp_array(img,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/img_transform.py:182: in warp_array

source_native_xy = mesh_projection(source_proj, nx, ny,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/img_transform.py:79: in mesh_projection

y, ystep = np.linspace(y_lower, y_upper, ny, retstep=True,

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/numpy/_core/function_base.py:189: in linspace

y = conv.wrap(y.astype(dtype, copy=False))

../xarray/xarray/core/dataarray.py:4674: in array_wrap

new_var = self.variable.array_wrap(obj, context)

../xarray/xarray/core/variable.py:2290: in array_wrap

return Variable(self.dims, obj)

../xarray/xarray/core/variable.py:397: in init

super().init(

../xarray/xarray/namedarray/core.py:264: in init

self._dims = self._parse_dimensions(dims)

self = <[AttributeError("'Variable' object has no attribute '_dims'") raised in repr()] Variable object at 0x31f3c03a0>, dims = ()
def _parse_dimensions(self, dims: _DimsLike) -> _Dims:
    dims = (dims,) if isinstance(dims, str) else tuple(dims)
    if len(dims) != self.ndim:


      raise ValueError(


            f"dimensions {dims} must have the same length as the "
            f"number of data dimensions, ndim={self.ndim}"
        )

E           ValueError: dimensions () must have the same length as the number of data dimensions, ndim=1
../xarray/xarray/namedarray/core.py:490: ValueError

Two more that are coming from Cartopy 0.23:

______________________________________________________________________________________________________________ test_lcc_minimal _______________________________________________________________________________________________________________
def test_lcc_minimal():
    """Test handling lambert conformal conic projection with minimal attributes."""
    attrs = {'grid_mapping_name': 'lambert_conformal_conic'}


  crs = CFProjection(attrs).to_cartopy()


tests/plots/test_mapping.py:153:

src/metpy/plots/mapping.py:89: in to_cartopy

return proj_handler(self._attrs, globe)

src/metpy/plots/mapping.py:148: in make_lcc

return ccrs.LambertConformal(globe=globe, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/crs.py:1795: in init

self._boundary = sgeom.LinearRing(points)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/geometry/polygon.py:104: in new

geom = shapely.linearrings(coordinates)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/decorators.py:77: in wrapped

return func(*args, **kwargs)

coords = array([[               nan,                nan,                nan],

[-16165309.26647638,  14814526.9736842 ,  ...4047416,  14776231.67371788,         0.        ],

[               nan,                nan,                nan]])

y = None, z = None, indices = None, out = None, kwargs = {}
@multithreading_enabled
def linearrings(coords, y=None, z=None, indices=None, out=None, **kwargs):
    """Create an array of linearrings.

    If the provided coords do not constitute a closed linestring, or if there
    are only 3 provided coords, the first
    coordinate is duplicated at the end to close the ring. This function will
    raise an exception if a linearring contains less than three points or if
    the terminal coordinates contain NaN (not-a-number).

    Parameters
    ----------
    coords : array_like
        An array of lists of coordinate tuples (2- or 3-dimensional) or, if ``y``
        is provided, an array of lists of x coordinates
    y : array_like, optional
    z : array_like, optional
    indices : array_like, optional
        Indices into the target array where input coordinates belong. If
        provided, the coords should be 2D with shape (N, 2) or (N, 3) and
        indices should be an array of shape (N,) with integers in increasing
        order. Missing indices result in a ValueError unless ``out`` is
        provided, in which case the original value in ``out`` is kept.
    out : ndarray, optional
        An array (with dtype object) to output the geometries into.
    **kwargs
        See :ref:`NumPy ufunc docs <ufuncs.kwargs>` for other keyword arguments.
        Ignored if ``indices`` is provided.

    See also
    --------
    linestrings

    Examples
    --------
    >>> linearrings([[0, 0], [0, 1], [1, 1], [0, 0]])
    <LINEARRING (0 0, 0 1, 1 1, 0 0)>
    >>> linearrings([[0, 0], [0, 1], [1, 1]])
    <LINEARRING (0 0, 0 1, 1 1, 0 0)>

    Notes
    -----
    - Usage of the ``y`` and ``z`` arguments will prevents lazy evaluation in ``dask``.
      Instead provide the coordinates as a ``(..., 2)`` or ``(..., 3)`` array using only ``coords``.
    """
    coords = _xyz_to_coords(coords, y, z)
    if indices is None:


      return lib.linearrings(coords, out=out, **kwargs)


E           shapely.errors.GEOSException: IllegalArgumentException: Points of LinearRing do not form a closed linestring
../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/creation.py:171: GEOSException

________________________________________________________________________________________________________ test_lcc_single_std_parallel _________________________________________________________________________________________________________
def test_lcc_single_std_parallel():
    """Test lambert conformal projection with one standard parallel."""
    attrs = {'grid_mapping_name': 'lambert_conformal_conic', 'standard_parallel': 25}


  crs = CFProjection(attrs).to_cartopy()


tests/plots/test_mapping.py:160:

src/metpy/plots/mapping.py:89: in to_cartopy

return proj_handler(self._attrs, globe)

src/metpy/plots/mapping.py:148: in make_lcc

return ccrs.LambertConformal(globe=globe, **kwargs)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/cartopy/crs.py:1795: in init

self._boundary = sgeom.LinearRing(points)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/geometry/polygon.py:104: in new

geom = shapely.linearrings(coordinates)

../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/decorators.py:77: in wrapped

return func(*args, **kwargs)

coords = array([[            nan,             nan,             nan],

[-2.02394033e+07,  7.05542455e+06,  0.00000000e+00]...      [ 2.02313877e+07,  7.02330796e+06,  0.00000000e+00],

[            nan,             nan,             nan]])

y = None, z = None, indices = None, out = None, kwargs = {}
@multithreading_enabled
def linearrings(coords, y=None, z=None, indices=None, out=None, **kwargs):
    """Create an array of linearrings.

    If the provided coords do not constitute a closed linestring, or if there
    are only 3 provided coords, the first
    coordinate is duplicated at the end to close the ring. This function will
    raise an exception if a linearring contains less than three points or if
    the terminal coordinates contain NaN (not-a-number).

    Parameters
    ----------
    coords : array_like
        An array of lists of coordinate tuples (2- or 3-dimensional) or, if ``y``
        is provided, an array of lists of x coordinates
    y : array_like, optional
    z : array_like, optional
    indices : array_like, optional
        Indices into the target array where input coordinates belong. If
        provided, the coords should be 2D with shape (N, 2) or (N, 3) and
        indices should be an array of shape (N,) with integers in increasing
        order. Missing indices result in a ValueError unless ``out`` is
        provided, in which case the original value in ``out`` is kept.
    out : ndarray, optional
        An array (with dtype object) to output the geometries into.
    **kwargs
        See :ref:`NumPy ufunc docs <ufuncs.kwargs>` for other keyword arguments.
        Ignored if ``indices`` is provided.

    See also
    --------
    linestrings

    Examples
    --------
    >>> linearrings([[0, 0], [0, 1], [1, 1], [0, 0]])
    <LINEARRING (0 0, 0 1, 1 1, 0 0)>
    >>> linearrings([[0, 0], [0, 1], [1, 1]])
    <LINEARRING (0 0, 0 1, 1 1, 0 0)>

    Notes
    -----
    - Usage of the ``y`` and ``z`` arguments will prevents lazy evaluation in ``dask``.
      Instead provide the coordinates as a ``(..., 2)`` or ``(..., 3)`` array using only ``coords``.
    """
    coords = _xyz_to_coords(coords, y, z)
    if indices is None:


      return lib.linearrings(coords, out=out, **kwargs)


E           shapely.errors.GEOSException: IllegalArgumentException: Points of LinearRing do not form a closed linestring
../../miniconda3/envs/numpy-dev-pip/lib/python3.12/site-packages/shapely/creation.py:171: GEOSException

I'll hit the latter 2 in #3474/upstream PR. Not sure where the problem exists for the first two yet.

This has been renamed to `trapezoid`, so do a conditional import to avoid a warning. This can be cleaned up in a couple years.

This has been moved from the (now) deprecated numeric namespace to an array_utils namespace. Apparently this wasn't a public API before (?) but now is.

We were a bit too stringent here, since versions can include alpha/beta as well as post versions. This was broken when numpy 2.0.0rc1 was installed.

This symbol is available at the top level of the module, so just use it there--just like the rest of the uses of this function in metpy. The full path here no longer works with numpy>=2.

This was causing a conflict between linewidth and linewidths that matplotlib 3.9.0rc2 caught and errored out on.

dopplershift · 2024-04-29T16:33:18Z

@dcamron I think I'm done making changes here so this is good for review/merge whenever you have a chance.

dopplershift requested review from kgoebber and a team as code owners April 26, 2024 21:24

dopplershift requested review from dcamron and removed request for a team April 26, 2024 21:24

dopplershift had a problem deploying to CodeCov April 26, 2024 21:34 — with GitHub Actions Failure

dopplershift had a problem deploying to CodeCov April 26, 2024 21:38 — with GitHub Actions Failure

dopplershift added 5 commits April 26, 2024 15:41

MNT: Update use of trapz for numpy>=2

1a9ba14

This has been renamed to `trapezoid`, so do a conditional import to avoid a warning. This can be cleaned up in a couple years.

MNT: Update normalize_axis_index for numpy>=2

37be3c3

This has been moved from the (now) deprecated numeric namespace to an array_utils namespace. Apparently this wasn't a public API before (?) but now is.

BUG: Fix version parsing for RC's, etc.

11f952f

We were a bit too stringent here, since versions can include alpha/beta as well as post versions. This was broken when numpy 2.0.0rc1 was installed.

MNT: Fix location of broadcast_to()

b94ac53

This symbol is available at the top level of the module, so just use it there--just like the rest of the uses of this function in metpy. The full path here no longer works with numpy>=2.

BUG: Fix kwarg in test

941109a

This was causing a conflict between linewidth and linewidths that matplotlib 3.9.0rc2 caught and errored out on.

dopplershift force-pushed the numpy2 branch from 05b9ce0 to 941109a Compare April 26, 2024 21:41

dopplershift temporarily deployed to CodeCov April 26, 2024 21:52 — with GitHub Actions Inactive

dopplershift temporarily deployed to CodeCov April 26, 2024 21:56 — with GitHub Actions Inactive

dcamron approved these changes May 10, 2024

View reviewed changes

dcamron merged commit 3e9fbb1 into Unidata:main May 10, 2024
34 checks passed

github-actions bot added this to the 1.7.0 milestone May 10, 2024

dopplershift deleted the numpy2 branch May 13, 2024 15:06

dopplershift modified the milestones: 1.7.0, 1.6.3 Jun 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Numpy 2 compatibility #3491

Numpy 2 compatibility #3491

dopplershift commented Apr 26, 2024 •

edited

Loading

dopplershift commented Apr 29, 2024

Numpy 2 compatibility #3491

Numpy 2 compatibility #3491

Conversation

dopplershift commented Apr 26, 2024 • edited Loading

Description Of Changes

dopplershift commented Apr 29, 2024

dopplershift commented Apr 26, 2024 •

edited

Loading