Test failures with PROJ 9.0.1 #2051

sebastic · 2022-06-08T18:17:07Z

Description

The first release candidates for PROJ 9.0.1 have been released, the release contains changes that cause test failures due to almost_equal tests failing.


=================================== FAILURES ===================================
______________________________ test_default[land] ______________________________

emphasis = 'land'

    @pytest.mark.parametrize("emphasis", ["land", "ocean"])
    def test_default(emphasis):
        igh = ccrs.InterruptedGoodeHomolosine(emphasis=emphasis)
        other_args = {"ellps=WGS84", "lon_0=0"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
        assert_almost_equal(
            np.array(igh.x_limits), [-20037508.3427892, 20037508.3427892]
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:30: AssertionError
_____________________________ test_default[ocean] ______________________________

emphasis = 'ocean'

    @pytest.mark.parametrize("emphasis", ["land", "ocean"])
    def test_default(emphasis):
        igh = ccrs.InterruptedGoodeHomolosine(emphasis=emphasis)
        other_args = {"ellps=WGS84", "lon_0=0"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
        assert_almost_equal(
            np.array(igh.x_limits), [-20037508.3427892, 20037508.3427892]
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:30: AssertionError
______________________ test_central_longitude[land--10.0] ______________________

emphasis = 'land', lon = -10.0

    @pytest.mark.parametrize(
        ("emphasis", "lon"),
        [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
    )
    def test_central_longitude(emphasis, lon):
        igh = ccrs.InterruptedGoodeHomolosine(
            central_longitude=lon, emphasis=emphasis
        )
        other_args = {"ellps=WGS84", f"lon_0={lon}"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
    
        assert_almost_equal(
            np.array(igh.x_limits),
            [-20037508.3427892, 20037508.3427892],
            decimal=5,
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:68: AssertionError
______________________ test_central_longitude[land-10.0] _______________________

emphasis = 'land', lon = 10.0

    @pytest.mark.parametrize(
        ("emphasis", "lon"),
        [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
    )
    def test_central_longitude(emphasis, lon):
        igh = ccrs.InterruptedGoodeHomolosine(
            central_longitude=lon, emphasis=emphasis
        )
        other_args = {"ellps=WGS84", f"lon_0={lon}"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
    
        assert_almost_equal(
            np.array(igh.x_limits),
            [-20037508.3427892, 20037508.3427892],
            decimal=5,
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:68: AssertionError
_____________________ test_central_longitude[ocean--10.0] ______________________

emphasis = 'ocean', lon = -10.0

    @pytest.mark.parametrize(
        ("emphasis", "lon"),
        [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
    )
    def test_central_longitude(emphasis, lon):
        igh = ccrs.InterruptedGoodeHomolosine(
            central_longitude=lon, emphasis=emphasis
        )
        other_args = {"ellps=WGS84", f"lon_0={lon}"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
    
        assert_almost_equal(
            np.array(igh.x_limits),
            [-20037508.3427892, 20037508.3427892],
            decimal=5,
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:68: AssertionError
______________________ test_central_longitude[ocean-10.0] ______________________

emphasis = 'ocean', lon = 10.0

    @pytest.mark.parametrize(
        ("emphasis", "lon"),
        [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
    )
    def test_central_longitude(emphasis, lon):
        igh = ccrs.InterruptedGoodeHomolosine(
            central_longitude=lon, emphasis=emphasis
        )
        other_args = {"ellps=WGS84", f"lon_0={lon}"}
        if emphasis == "land":
            check_proj_params("igh", igh, other_args)
        elif emphasis == "ocean":
            check_proj_params("igh_o", igh, other_args)
    
        assert_almost_equal(
            np.array(igh.x_limits),
            [-20037508.3427892, 20037508.3427892],
            decimal=5,
        )
>       assert_almost_equal(
            np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
        )
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017592
E       Max relative difference: 2.02598163e-11
E        x: array([-8683259.7162588,  8683259.7162588])
E        y: array([-8683259.7164347,  8683259.7164347])

cartopy/tests/crs/test_interrupted_goode_homolosine.py:68: AssertionError
_________________________________ test_default _________________________________

    def test_default():
        moll = ccrs.Mollweide()
        other_args = {'a=6378137.0', 'lon_0=0'}
        check_proj_params('moll', moll, other_args)
    
        assert_almost_equal(np.array(moll.x_limits),
                            [-18040095.6961473, 18040095.6961473])
>       assert_almost_equal(np.array(moll.y_limits),
                            [-9020047.8480736, 9020047.8480736])
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017584
E       Max relative difference: 1.94946876e-11
E        x: array([-9020047.8478978,  9020047.8478978])
E        y: array([-9020047.8480736,  9020047.8480736])

cartopy/tests/crs/test_mollweide.py:26: AssertionError
______________________________ test_ellipse_globe ______________________________

    def test_ellipse_globe():
        globe = ccrs.Globe(ellipse='WGS84')
        with pytest.warns(UserWarning,
                          match='does not handle elliptical globes.') as w:
            moll = ccrs.Mollweide(globe=globe)
            assert len(w) == 1
    
        other_args = {'ellps=WGS84', 'lon_0=0'}
        check_proj_params('moll', moll, other_args)
    
        # Limits are the same as default since ellipses are not supported.
        assert_almost_equal(moll.x_limits, [-18040095.6961473, 18040095.6961473])
>       assert_almost_equal(moll.y_limits, [-9020047.8480736, 9020047.8480736])
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017584
E       Max relative difference: 1.94946876e-11
E        x: array([-9020047.8478978,  9020047.8478978])
E        y: array([-9020047.8480736,  9020047.8480736])

cartopy/tests/crs/test_mollweide.py:52: AssertionError
________________________ test_central_longitude[-10.0] _________________________

lon = -10.0

    @pytest.mark.parametrize('lon', [-10.0, 10.0])
    def test_central_longitude(lon):
        moll = ccrs.Mollweide(central_longitude=lon)
        other_args = {'a=6378137.0', f'lon_0={lon}'}
        check_proj_params('moll', moll, other_args)
    
        assert_almost_equal(np.array(moll.x_limits),
                            [-18040095.6961473, 18040095.6961473],
                            decimal=5)
>       assert_almost_equal(np.array(moll.y_limits),
                            [-9020047.8480736, 9020047.8480736])
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017584
E       Max relative difference: 1.94946876e-11
E        x: array([-9020047.8478978,  9020047.8478978])
E        y: array([-9020047.8480736,  9020047.8480736])

cartopy/tests/crs/test_mollweide.py:89: AssertionError
_________________________ test_central_longitude[10.0] _________________________

lon = 10.0

    @pytest.mark.parametrize('lon', [-10.0, 10.0])
    def test_central_longitude(lon):
        moll = ccrs.Mollweide(central_longitude=lon)
        other_args = {'a=6378137.0', f'lon_0={lon}'}
        check_proj_params('moll', moll, other_args)
    
        assert_almost_equal(np.array(moll.x_limits),
                            [-18040095.6961473, 18040095.6961473],
                            decimal=5)
>       assert_almost_equal(np.array(moll.y_limits),
                            [-9020047.8480736, 9020047.8480736])
E       AssertionError: 
E       Arrays are not almost equal to 7 decimals
E       
E       Mismatched elements: 2 / 2 (100%)
E       Max absolute difference: 0.00017584
E       Max relative difference: 1.94946876e-11
E        x: array([-9020047.8478978,  9020047.8478978])
E        y: array([-9020047.8480736,  9020047.8480736])

cartopy/tests/crs/test_mollweide.py:89: AssertionError
=============================== warnings summary ===============================
.pybuild/cpython3_3.10_cartopy/build/cartopy/tests/test_crs.py::TestCRS::test_osgb[True]
.pybuild/cpython3_3.10_cartopy/build/cartopy/tests/test_crs.py::TestCRS::test_osgb[False]
.pybuild/cpython3_3.10_cartopy/build/cartopy/tests/test_crs.py::TestCRS::test_epsg
  /build/python-cartopy-0.20.2+dfsg/.pybuild/cpython3_3.10_cartopy/build/cartopy/tests/test_crs.py:61: UserWarning: uk_os_OSTN15_NTv2_OSGBtoETRS.tif is unavailable; testing OSGB at reduced precision
    warnings.warn(f'{grid_name} is unavailable; '

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_default[land]
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_default[ocean]
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_central_longitude[land--10.0]
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_central_longitude[land-10.0]
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_central_longitude[ocean--10.0]
FAILED cartopy/tests/crs/test_interrupted_goode_homolosine.py::test_central_longitude[ocean-10.0]
FAILED cartopy/tests/crs/test_mollweide.py::test_default - AssertionError: 
FAILED cartopy/tests/crs/test_mollweide.py::test_ellipse_globe - AssertionErr...
FAILED cartopy/tests/crs/test_mollweide.py::test_central_longitude[-10.0] - A...
FAILED cartopy/tests/crs/test_mollweide.py::test_central_longitude[10.0] - As...
= 10 failed, 533 passed, 3 skipped, 121 deselected, 2 xfailed, 2 xpassed, 3 warnings in 27.66s =

Code to reproduce

Build cartopy with PROJ 9.0.1RC2.

The following changes work around the test failures:

--- a/lib/cartopy/tests/crs/test_interrupted_goode_homolosine.py
+++ b/lib/cartopy/tests/crs/test_interrupted_goode_homolosine.py
@@ -16,6 +16,7 @@ import cartopy.crs as ccrs
 from .helpers import check_proj_params
 
 
+@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize("emphasis", ["land", "ocean"])
 def test_default(emphasis):
     igh = ccrs.InterruptedGoodeHomolosine(emphasis=emphasis)
@@ -46,6 +47,7 @@ def test_eccentric_globe(emphasis):
     assert_almost_equal(np.array(igh.y_limits), [-1361.410035, 1361.410035])
 
 
+@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize(
     ("emphasis", "lon"),
     [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
--- a/lib/cartopy/tests/crs/test_mollweide.py
+++ b/lib/cartopy/tests/crs/test_mollweide.py
@@ -16,6 +16,7 @@ import cartopy.crs as ccrs
 from .helpers import check_proj_params
 
 
+@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 def test_default():
     moll = ccrs.Mollweide()
     other_args = {'a=6378137.0', 'lon_0=0'}
@@ -37,6 +38,7 @@ def test_sphere_globe():
     assert_almost_equal(moll.y_limits, [-1414.2135624, 1414.2135624])
 
 
+@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 def test_ellipse_globe():
     globe = ccrs.Globe(ellipse='WGS84')
     with pytest.warns(UserWarning,
@@ -77,6 +79,7 @@ def test_offset():
     assert tuple(np.array(crs.y_limits) - 4321) == crs_offset.y_limits
 
 
+@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize('lon', [-10.0, 10.0])
 def test_central_longitude(lon):
     moll = ccrs.Mollweide(central_longitude=lon)

The text was updated successfully, but these errors were encountered:

dopplershift · 2022-06-08T20:20:09Z

I'd be a lot more interested in knowing how those tests are failing exactly. Are these tests overly precise? Were the values wrong before?

sebastic · 2022-06-09T03:15:14Z

Overly precise is a likely cause. The values could also be wrong, someone who understands the CRS changes will need to answer that.

rcomer · 2022-06-09T07:35:34Z

I wonder what would happen if you replace assert_almost_equal with assert_allclose. The former is not really recommended any more: see note at https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_almost_equal.html

Edit: and the relative differences now showing the OP really are tiny!

sebastic · 2022-06-09T09:31:24Z

With these changes (on top of the xfail changes), the tests succeed with PROJ 9.0.1:

--- a/lib/cartopy/tests/crs/test_interrupted_goode_homolosine.py
+++ b/lib/cartopy/tests/crs/test_interrupted_goode_homolosine.py
@@ -9,14 +9,13 @@ Tests for the InterruptedGoodeHomolosine
 """
 
 import numpy as np
-from numpy.testing import assert_almost_equal
+from numpy.testing import assert_almost_equal, assert_allclose
 import pytest
 
 import cartopy.crs as ccrs
 from .helpers import check_proj_params
 
 
-@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize("emphasis", ["land", "ocean"])
 def test_default(emphasis):
     igh = ccrs.InterruptedGoodeHomolosine(emphasis=emphasis)
@@ -25,10 +24,10 @@ def test_default(emphasis):
         check_proj_params("igh", igh, other_args)
     elif emphasis == "ocean":
         check_proj_params("igh_o", igh, other_args)
-    assert_almost_equal(
+    assert_allclose(
         np.array(igh.x_limits), [-20037508.3427892, 20037508.3427892]
     )
-    assert_almost_equal(
+    assert_allclose(
         np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
     )
 
@@ -47,7 +46,6 @@ def test_eccentric_globe(emphasis):
     assert_almost_equal(np.array(igh.y_limits), [-1361.410035, 1361.410035])
 
 
-@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize(
     ("emphasis", "lon"),
     [("land", -10.0), ("land", 10.0), ("ocean", -10.0), ("ocean", 10.0)],
@@ -62,11 +60,10 @@ def test_central_longitude(emphasis, lon
     elif emphasis == "ocean":
         check_proj_params("igh_o", igh, other_args)
 
-    assert_almost_equal(
+    assert_allclose(
         np.array(igh.x_limits),
         [-20037508.3427892, 20037508.3427892],
-        decimal=5,
     )
-    assert_almost_equal(
+    assert_allclose(
         np.array(igh.y_limits), [-8683259.7164347, 8683259.7164347]
     )
--- a/lib/cartopy/tests/crs/test_mollweide.py
+++ b/lib/cartopy/tests/crs/test_mollweide.py
@@ -9,22 +9,21 @@ Tests for the Mollweide coordinate syste
 """
 
 import numpy as np
-from numpy.testing import assert_almost_equal
+from numpy.testing import assert_almost_equal, assert_allclose
 import pytest
 
 import cartopy.crs as ccrs
 from .helpers import check_proj_params
 
 
-@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 def test_default():
     moll = ccrs.Mollweide()
     other_args = {'a=6378137.0', 'lon_0=0'}
     check_proj_params('moll', moll, other_args)
 
-    assert_almost_equal(np.array(moll.x_limits),
+    assert_allclose(np.array(moll.x_limits),
                         [-18040095.6961473, 18040095.6961473])
-    assert_almost_equal(np.array(moll.y_limits),
+    assert_allclose(np.array(moll.y_limits),
                         [-9020047.8480736, 9020047.8480736])
 
 
@@ -38,7 +37,6 @@ def test_sphere_globe():
     assert_almost_equal(moll.y_limits, [-1414.2135624, 1414.2135624])
 
 
-@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 def test_ellipse_globe():
     globe = ccrs.Globe(ellipse='WGS84')
     with pytest.warns(UserWarning,
@@ -50,8 +48,8 @@ def test_ellipse_globe():
     check_proj_params('moll', moll, other_args)
 
     # Limits are the same as default since ellipses are not supported.
-    assert_almost_equal(moll.x_limits, [-18040095.6961473, 18040095.6961473])
-    assert_almost_equal(moll.y_limits, [-9020047.8480736, 9020047.8480736])
+    assert_allclose(moll.x_limits, [-18040095.6961473, 18040095.6961473])
+    assert_allclose(moll.y_limits, [-9020047.8480736, 9020047.8480736])
 
 
 def test_eccentric_globe():
@@ -79,17 +77,15 @@ def test_offset():
     assert tuple(np.array(crs.y_limits) - 4321) == crs_offset.y_limits
 
 
-@pytest.mark.xfail(reason='Fails with PROJ 9.0.1', strict=False)
 @pytest.mark.parametrize('lon', [-10.0, 10.0])
 def test_central_longitude(lon):
     moll = ccrs.Mollweide(central_longitude=lon)
     other_args = {'a=6378137.0', f'lon_0={lon}'}
     check_proj_params('moll', moll, other_args)
 
-    assert_almost_equal(np.array(moll.x_limits),
-                        [-18040095.6961473, 18040095.6961473],
-                        decimal=5)
-    assert_almost_equal(np.array(moll.y_limits),
+    assert_allclose(np.array(moll.x_limits),
+                        [-18040095.6961473, 18040095.6961473])
+    assert_allclose(np.array(moll.y_limits),
                         [-9020047.8480736, 9020047.8480736])

greglucas · 2022-06-09T12:59:29Z

Our test limits are somewhat arbitrarily set. The assert_allclose method definitely seem like the way to go here.

greglucas added Experience-needed: low Component: proj Type: Infrastructure labels Jun 9, 2022

greglucas added this to the 0.21 milestone Jun 9, 2022

greglucas mentioned this issue Jun 27, 2022

TST: Change testing method for tolerances #2056

Merged

QuLogic closed this as completed in #2056 Jun 28, 2022

QuLogic mentioned this issue Jun 29, 2022

Backports for 0.20.x #2059

Merged

QuLogic modified the milestones: 0.21, 0.20.3 Jun 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failures with PROJ 9.0.1 #2051

Test failures with PROJ 9.0.1 #2051

sebastic commented Jun 8, 2022 •

edited

Loading

dopplershift commented Jun 8, 2022

sebastic commented Jun 9, 2022

rcomer commented Jun 9, 2022 •

edited

Loading

sebastic commented Jun 9, 2022

greglucas commented Jun 9, 2022

Test failures with PROJ 9.0.1 #2051

Test failures with PROJ 9.0.1 #2051

Comments

sebastic commented Jun 8, 2022 • edited Loading

Description

Code to reproduce

dopplershift commented Jun 8, 2022

sebastic commented Jun 9, 2022

rcomer commented Jun 9, 2022 • edited Loading

sebastic commented Jun 9, 2022

greglucas commented Jun 9, 2022

sebastic commented Jun 8, 2022 •

edited

Loading

rcomer commented Jun 9, 2022 •

edited

Loading