Drop EOL Python, simplify PyPy situation, improve testing #74

h-vetinari · 2023-06-23T02:09:23Z

Addressing a couple of inconsistencies & clean-ups:

we are mixing availability of wheels with compatibility (inconsistently; e.g. there are no wheels for s390x or loongarch64)
we occasionally produce more than one pin for invalid platform combinations
we are carrying around cruft from python versions that are long dead

What I'm doing:

drop old Python
reflect compatibility not just on availability of wheels (PyPy, ppc, etc.)
substantially expand testing, becoming strict on all valid platform combinations in the expectation of zero or one pins
remove all skips to also test invalid platform combinations (so that we test at least there's maximum 1 pin)
the test module become a much more easily readable documentation of what setup.cfg is supposed to reflect
reflect the existence of win-arm64 & linux-ppc64le

What should still be improved:

we should make it clear on which architectures AIX/OS400 is supported. I'm now using "x86", "x86_64", "s390x", which is broadly equivalent to what it was set to before (though I removed arm64 from that list, because it's +/- equivalent to aarch64 that was already marked as invalid)

Closes #70
Closes #71
Closes #72

h-vetinari

Some notes for ease of reviewing (though I recommend to go through it per commit)

.github/workflows/main.yml

tests/conftest.py

tests/test_dependencies.py

h-vetinari · 2023-06-23T03:00:16Z

tests/test_dependencies.py

+    # since we cannot run the installation tests on all platforms,
+    # we formulate the conditions when we expect a pin
+    expect_pin = False
+    match (platform_system, platform_machine):
+        case [("Linux" | "Darwin" | "Windows"), "x86_64"]:
+            expect_pin = True  # baseline
+        case [("Linux" | "Windows"), "x86"]:
+            # 32 bit wheels on Linux only until numpy 1.21, but should still be compatible
+            expect_pin = True
+        case ["Linux", "aarch64"]:
+            expect_pin = True  # as of 1.19.2
+        case ["Darwin", "arm64"]:
+            expect_pin = True  # as of 1.21
+        # no official wheels, but register minimum compatibility
+        case ["Linux", "ppc64le"]:
+            expect_pin = True  # support +/- equivalent to Linux/aarch64, but no wheels
+        case ["Linux", "s390x"]:
+            expect_pin = True  # as of 1.17.5
+        case ["Linux", "loongarch64"]:
+            expect_pin = True  # as of 1.22


I find this a very compact and readable crystallization of what we're trying to achieve in the setup.cfg, in addition to the distinction of whether wheels are available or not (we could add more testing for this if we whip up some emulation, see #73).

tests/test_dependencies.py

setup.cfg

rgommers

Thanks @h-vetinari. This looks quite good overall. The testing changes seem helpful; I didn't review them in a lot of detail, but they're quite understandable and I'll do a bit more checking before merging.

setup.cfg

h-vetinari · 2023-06-30T11:31:42Z

Thanks @h-vetinari. This looks quite good overall.

I switched this now to use the default pins for linux-ppc & win-arm64. However, I'm still proposing to not distinguish PyPy from CPython for language level >=3.9. Otherwise I think I've responded to all review feedback now.

PS. I rebased & refactored the commits to have a proper history (for archeology, potential reverts, etc.), IMO this should not be squash-merged.

numpy has been passing the test suite on PyPy 3.8 & 3.9 since at least 1.19. There are no wheels for PyPy 3.9 until numpy 1.25, but neither are there official wheels for loongarch64 etc., so we should reflect that PyPy is supported. Future PyPy versions will only be more compatible with the respective CPython version, so it's not a stretch to mirror the constraints we have for (say) CPython 3.11 to those for a future PyPy 3.11.

… CI)

…st 1 pin

Co-Authored-By: Randy Döring <[email protected]>

rgommers

Thanks @h-vetinari.

However, I'm still proposing to not distinguish PyPy from CPython for language level >=3.9. Otherwise I think I've responded to all review feedback now.

Changing this kind of thing without a compelling reason seems potentially risky to me, without much upside. However, it's likely we'll have to change it again either way when a next PyPy version is released. So let's go with this and if anyone complains, we can revert.

In it goes!

rgommers · 2023-08-03T16:18:53Z

Cut a release: https://pypi.org/project/oldest-supported-numpy/2023.8.3/#files

h-vetinari force-pushed the simplify branch 2 times, most recently from f9f361c to e204ff0 Compare June 23, 2023 02:45

h-vetinari mentioned this pull request Jun 23, 2023

Fix incompatible requirements #71

Closed

h-vetinari commented Jun 23, 2023

View reviewed changes

radoering reviewed Jun 23, 2023

View reviewed changes

setup.cfg Outdated Show resolved Hide resolved

radoering mentioned this pull request Jun 24, 2023

full support for duplicate dependencies with overlapping markers python-poetry/poetry#7257

Merged

2 tasks

h-vetinari mentioned this pull request Jun 28, 2023

BUG: Update oldest-supported-numpy metadata scipy/scipy#18773

Closed

rgommers reviewed Jun 28, 2023

View reviewed changes

setup.cfg Show resolved Hide resolved

setup.cfg Outdated Show resolved Hide resolved

setup.cfg Outdated Show resolved Hide resolved

h-vetinari force-pushed the simplify branch 2 times, most recently from 76eccc3 to adf7141 Compare June 30, 2023 10:20

h-vetinari and others added 12 commits June 30, 2023 22:44

remove EOL python versions

c47c7e0

add a .gitignore to ignore artefacts from local testing

bc41e0d

improve skips for PyPy

2bfba0f

improve skipping for incompatible system/machine combinations

84ed4e5

improve testing for cases where where we expect a pin (but don't have…

2e25b69

… CI)

reflect existence of win-arm64 as a platform

c096b82

add metadata for compatibility with ppc

ec378cd

turn get_package_dependencies() into fixture

b611396

split installation test out from where we use py310 syntax

d3473d8

don't skip tests for invalid combinations; ensure they too have at mo…

e718d93

…st 1 pin

refine loongarch pin to apply only for py>=3.8

0457715

Co-Authored-By: Randy Döring <[email protected]>

h-vetinari force-pushed the simplify branch from 305fe1b to 0457715 Compare June 30, 2023 11:44

rgommers approved these changes Aug 3, 2023

View reviewed changes

rgommers merged commit 13bed2b into scipy:main Aug 3, 2023
21 checks passed

h-vetinari deleted the simplify branch August 4, 2023 00:04

h-vetinari mentioned this pull request Aug 4, 2023

MNT: add pins for pypy39 #78

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop EOL Python, simplify PyPy situation, improve testing #74

Drop EOL Python, simplify PyPy situation, improve testing #74

h-vetinari commented Jun 23, 2023 •

edited

Loading

h-vetinari left a comment •

edited

Loading

h-vetinari Jun 23, 2023

rgommers left a comment

h-vetinari commented Jun 30, 2023 •

edited

Loading

rgommers left a comment

rgommers commented Aug 3, 2023

Drop EOL Python, simplify PyPy situation, improve testing #74

Drop EOL Python, simplify PyPy situation, improve testing #74

Conversation

h-vetinari commented Jun 23, 2023 • edited Loading

h-vetinari left a comment • edited Loading

Choose a reason for hiding this comment

h-vetinari Jun 23, 2023

Choose a reason for hiding this comment

rgommers left a comment

Choose a reason for hiding this comment

h-vetinari commented Jun 30, 2023 • edited Loading

rgommers left a comment

Choose a reason for hiding this comment

rgommers commented Aug 3, 2023

h-vetinari commented Jun 23, 2023 •

edited

Loading

h-vetinari left a comment •

edited

Loading

h-vetinari commented Jun 30, 2023 •

edited

Loading