Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pkgs/sagemath-standard: Move metadata from setup.cfg to pyproject.toml #36951

Merged
merged 12 commits into from
Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions bootstrap
Original file line number Diff line number Diff line change
Expand Up @@ -232,8 +232,7 @@ save () {
src/environment-optional-3.[89].yml src/environment-optional-3.1[0-9].yml \
src/Pipfile \
src/pyproject.toml \
src/requirements.txt \
src/setup.cfg
src/requirements.txt

# Update version
echo "$NEWCONFVERSION" >$PKG/package-version.txt
Expand Down
2 changes: 1 addition & 1 deletion build/bin/write-dockerfile.sh
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@ $ADD Makefile VERSION.txt COPYING.txt condarc.yml README.md bootstrap bootstrap-
$ADD config/config.rpath config/config.rpath
$ADD src/doc/bootstrap src/doc/bootstrap
$ADD src/bin src/bin
$ADD src/Pipfile.m4 src/pyproject.toml.m4 src/requirements.txt.m4 src/setup.cfg.m4 src/VERSION.txt src/
$ADD src/Pipfile.m4 src/pyproject.toml.m4 src/requirements.txt.m4 src/VERSION.txt src/
$ADD m4 ./m4
$ADD pkgs pkgs
$ADD build ./build
Expand Down
1 change: 0 additions & 1 deletion pkgs/sagemath-standard/setup.cfg

This file was deleted.

1 change: 0 additions & 1 deletion pkgs/sagemath-standard/setup.cfg.m4

This file was deleted.

4 changes: 2 additions & 2 deletions pkgs/sagemath-standard/tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
[tox]
envlist =
# Build and test without using the concrete dependencies specified by requirements.txt,
# using the dependencies declared in pyproject.toml and setup.cfg (install-requires) only:
# using the dependencies declared in pyproject.toml (build-system requires, dependencies) only:
# Still use ONLY the wheels built and stored by the Sage distribution (no PyPI).
#
# ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'
Expand Down Expand Up @@ -45,7 +45,7 @@ envlist =
#
# ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-pipenv-dist)'
#
# Build using the dependencies declared in pyproject.toml and setup.cfg (install-requires) only.
# Build using the dependencies declared in pyproject.toml (build-system requires, dependencies) only:
# Use the wheels built and stored by the Sage distribution,
# and additionally allow packages from PyPI.
#
Expand Down
8 changes: 4 additions & 4 deletions src/Pipfile.m4
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
## We do not list packages that are already declared as install_requires
## in setup.cfg
## We do not list packages that are already declared as dependencies (install_requires)
## in pyproject.toml
pycodestyle = "*"
tox = "*"
pytest = "*"
rope = "*"
six = "*"

[packages]
## We do not list packages that are already declared as install_requires
## in setup.cfg
## We do not list packages that are already declared as dependencies (install_requires)
## in pyproject.toml

[packages.e1839a8]
path = "."
4 changes: 2 additions & 2 deletions src/doc/en/developer/coding_basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,8 @@ of the following places:
file = os.path.join(SAGE_EXTCODE, 'directory', 'file')

In both cases, the files must be listed (explicitly or via wildcards) in
the section ``options.package_data`` of the file
``SAGE_ROOT/pkgs/sagemath-standard/setup.cfg.m4`` (or the corresponding
the section ``[tool.setuptools.package-data]`` of the file
``SAGE_ROOT/pkgs/sagemath-standard/pyproject.toml.m4`` (or the corresponding
file of another distribution).


Expand Down
5 changes: 3 additions & 2 deletions src/doc/en/developer/coding_in_python.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ Python language standard

Sage library code needs to be compatible with all versions of Python
that Sage supports. The information regarding the supported versions
can be found in the files ``build/pkgs/python3/spkg-configure.m4`` and
``src/setup.cfg.m4``.
can be found in the files ``build/pkgs/python3/spkg-configure.m4``
(for the Sage distribution), ``m4/pyproject_toml_metadata.m4`` (for
the Sage library and most other distribution packages in ``pkgs/``).

Python 3.9 is the oldest supported version. Hence,
all language and library features that are available in Python 3.9 can
Expand Down
25 changes: 14 additions & 11 deletions src/doc/en/developer/packaging_sage_library.rst
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,7 @@ The source directory of a distribution package, such as

$ ./sage --fixdistributions --set all --from-egg-info

- `pyproject.toml <https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/>`_,
`setup.cfg <https://setuptools.pypa.io/en/latest/userguide/declarative_config.html>`_,
- `pyproject.toml <https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/>`_
and `requirements.txt <https://pip.pypa.io/en/stable/user_guide/#requirements-files>`_ --
standard Python packaging metadata, declaring the distribution name, dependencies,
etc.
Expand Down Expand Up @@ -288,8 +287,9 @@ modules must be part of the distribution, or provided by another
distribution -- which then must be declared as a run-time dependency.

*Declaring run-time dependencies:* These dependencies are declared in
``setup.cfg`` (generated from ``setup.cfg.m4``) as
`install_requires <https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency>`_.
``pyproject.toml`` (generated from ``pyproject.toml.m4``) as
`[project] dependencies <https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#declaring-required-dependency>`_ (in the older terminology of ``setup.cfg`` and ``setup.py``,
these dependencies were known as ``install_requires``).

*Reducing module-level run-time dependencies:*

Expand Down Expand Up @@ -467,14 +467,17 @@ features, which will only be working if the user also has installed
**sagemath-symbolics**.

*Declaring optional run-time dependencies:* It is possible to declare
such optional dependencies as `extras_require <https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies>`_ in ``setup.cfg``
(generated from ``setup.cfg.m4``). This is a very limited mechanism
such dependencies as `[project.optional-dependencies] <https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies>`_ in ``pyproject.toml``
(generated from ``pyproject.toml.m4``).
(In the older terminology of ``setup.cfg`` and ``setup.py``,
these optional dependencies were known as ``extras_require``.)
This is a very limited mechanism
-- in particular it does not affect the build phase of the
distribution in any way. It basically only provides a way to give a
nickname to a distribution that can be installed as an add-on.

In our example, we could declare an ``extras_require`` so that users
could use ``pip install sagemath-coding[symbolics]``.
In our example, we could declare an optional dependency so that users
could use ``pip install "sagemath-coding[symbolics]"``.


Doctest-only dependencies
Expand Down Expand Up @@ -516,7 +519,7 @@ The version information for dependencies comes from the files
``build/pkgs/*/package-version.txt``. We use the
`m4 <https://www.gnu.org/software/m4/manual/html_node/index.html>`_
macro processor to insert the version information in the generated files
``pyproject.toml``, ``setup.cfg``, ``requirements.txt``.
``pyproject.toml`` and ``requirements.txt``.


Hierarchy of distribution packages
Expand Down Expand Up @@ -546,8 +549,8 @@ Hierarchy of distribution packages
sphinx_plot(g, figsize=(8, 4), axes=False)


Solid arrows indicate ``install_requires``, i.e., a declared runtime dependency.
Dashed arrows indicate ``extras_require``, i.e., a declared optional runtime dependency.
Solid arrows indicate declared runtime dependencies (``install_requires``).
Dashed arrows indicate declared optional runtime dependencies (``extras_require``).
Not shown in the diagram are build dependencies and optional dependencies for testing.

- `sage_conf <https://pypi.org/project/sage-conf/>`_ is a configuration
Expand Down
164 changes: 148 additions & 16 deletions src/pyproject.toml.m4
Original file line number Diff line number Diff line change
@@ -1,24 +1,156 @@
include(`sage_spkg_versions_toml.m4')dnl' -*- conf-toml -*-
[build-system]
# Minimum requirements for the build system to execute.
requires = [
"sage_setup[autogen]",
# Some version of sage-conf is required.
# Note that PEP517/518 have no notion of optional sage_spkg dependencies:
# https://github.com/pypa/pip/issues/6144
esyscmd(`sage-get-system-packages install-requires-toml \
sage_conf \
setuptools \
wheel \
sage_setup \
cypari \
cysignals \
cython \
gmpy2 \
jinja2 \
jupyter_core \
numpy \
pkgconfig \
pplpy \
memory_allocator \
')]
SPKG_INSTALL_REQUIRES_sage_conf
SPKG_INSTALL_REQUIRES_setuptools
SPKG_INSTALL_REQUIRES_wheel
SPKG_INSTALL_REQUIRES_sage_setup
SPKG_INSTALL_REQUIRES_cypari
SPKG_INSTALL_REQUIRES_cysignals
SPKG_INSTALL_REQUIRES_cython
SPKG_INSTALL_REQUIRES_gmpy2
SPKG_INSTALL_REQUIRES_jupyter_core
SPKG_INSTALL_REQUIRES_memory_allocator
SPKG_INSTALL_REQUIRES_numpy
SPKG_INSTALL_REQUIRES_pkgconfig
SPKG_INSTALL_REQUIRES_pplpy
Comment on lines 6 to +21
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a problem here. All of those are not build-system requirements. According to pep 517 this should only list packages that are needed to run the build-system (not to build a wheel).

Only sage-setup and setuptools are necessary to run the build system, and indeed they suffice to build a sdist.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I salute you for starting to read the specs.

But you're a little bit too quick to declare that this "needs work".

There is not "one true" declaration of build-system requirements.

As I explained to you elsewhere, setuptools does not support declarations of PEP-517/660 hooks (other backends do). When we decide to use setuptools as the PEP-517/660 build-backend, we work with what is supported. We have to build wheels, the wheels need all of these things, hence we declare them as build-system requirements.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides, the present PR does not even make a change to what is declared...

]
build-backend = "setuptools.build_meta"

[project]
name = "sagemath-standard"
description = "Sage: Open Source Mathematics Software: Standard Python Library"
dependencies = [
SPKG_INSTALL_REQUIRES_sage_conf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can simply use dynamic metadata and

dependencies = {file = ["build/pkgs/whatever/install-requires.txt", "build/pkgs/whatever2/install-requires.txt", ...]}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, but that would not be an improvement.
It would make the sdist of this distribution depend on the monorepo / the Sage distribution.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setuptools automatically includes the linked files (and only these) in the sdist.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These file paths build/pkgs/... do not exist in this directory.

SPKG_INSTALL_REQUIRES_six
dnl From build/pkgs/sagelib/dependencies
SPKG_INSTALL_REQUIRES_conway_polynomials
SPKG_INSTALL_REQUIRES_cypari
SPKG_INSTALL_REQUIRES_cysignals
SPKG_INSTALL_REQUIRES_cython
SPKG_INSTALL_REQUIRES_gmpy2
SPKG_INSTALL_REQUIRES_importlib_metadata
SPKG_INSTALL_REQUIRES_importlib_resources
SPKG_INSTALL_REQUIRES_jupyter_core
SPKG_INSTALL_REQUIRES_lrcalc_python
SPKG_INSTALL_REQUIRES_memory_allocator
SPKG_INSTALL_REQUIRES_numpy
SPKG_INSTALL_REQUIRES_pkgconfig
SPKG_INSTALL_REQUIRES_pplpy
SPKG_INSTALL_REQUIRES_primecountpy
SPKG_INSTALL_REQUIRES_requests
SPKG_INSTALL_REQUIRES_typing_extensions
dnl From Makefile.in: SAGERUNTIME
SPKG_INSTALL_REQUIRES_ipython
SPKG_INSTALL_REQUIRES_pexpect
dnl From Makefile.in: DOC_DEPENDENCIES
SPKG_INSTALL_REQUIRES_sphinx
SPKG_INSTALL_REQUIRES_networkx
SPKG_INSTALL_REQUIRES_scipy
SPKG_INSTALL_REQUIRES_sympy
SPKG_INSTALL_REQUIRES_matplotlib
SPKG_INSTALL_REQUIRES_pillow
SPKG_INSTALL_REQUIRES_mpmath
SPKG_INSTALL_REQUIRES_ipykernel
SPKG_INSTALL_REQUIRES_jupyter_client
SPKG_INSTALL_REQUIRES_ipywidgets
SPKG_INSTALL_REQUIRES_fpylll
dnl pycryptosat # Sage distribution installs it as part of cryptominisat. According to its README on https://pypi.org/project/pycryptosat/: "The pycryptosat python package compiles while compiling CryptoMiniSat. It cannot be compiled on its own, it must be compiled at the same time as CryptoMiniSat."
dnl Packages with important upper version bounds
SPKG_INSTALL_REQUIRES_ptyprocess
]
dynamic = ["version"]
include(`pyproject_toml_metadata.m4')dnl'

[project.optional-dependencies]
R = [
SPKG_INSTALL_REQUIRES_rpy2
]

[project.readme]
file = "README.rst"
content-type = "text/x-rst"

[tool.setuptools]
script-files = [
# The sage script
"bin/sage",
# Other scripts that should be in the path also for OS packaging of sage:
"bin/sage-eval",
# Included because it is useful for doctesting/coverage testing user scripts too:
"bin/sage-runtests",
"bin/sage-fixdoctests",
"bin/sage-coverage",
# The following is deprecated but might still be used in user package install scripts
"bin/sage-cython",
# Helper scripts invoked by sage script
# (they would actually belong to something like libexec)
"bin/sage-cachegrind",
"bin/sage-callgrind",
"bin/sage-massif",
"bin/sage-omega",
"bin/sage-valgrind",
"bin/sage-venv-config",
"bin/sage-version.sh",
"bin/sage-cleaner",
# Only makes sense in sage-the-distribution. TODO: Move to another installation script.
"bin/sage-list-packages",
# Uncategorized scripts in alphabetical order
"bin/math-readline",
"bin/sage-env",
# sage-env-config -- installed by sage_conf
# sage-env-config.in -- not to be installed
"bin/sage-grep",
"bin/sage-grepdoc",
"bin/sage-inline-fortran",
"bin/sage-ipynb2rst",
"bin/sage-ipython",
"bin/sage-notebook",
"bin/sage-num-threads.py",
"bin/sage-preparse",
"bin/sage-python",
"bin/sage-run",
"bin/sage-run-cython",
"bin/sage-startuptime.py",
"bin/sage-update-version",
]
license-files = ["LICENSE.txt"]
include-package-data = false

[tool.setuptools.package-data]
"sage.libs.gap" = ["sage.gaprc"]
"sage.interfaces" = ["sage-maxima.lisp"]
"sage.doctest" = ["tests/*"]
"sage.repl.rich_output" = ["example*"]
sage = [
"ext_data/*",
"ext_data/kenzo/*",
"ext_data/singular/*",
"ext_data/singular/function_field/*",
"ext_data/images/*",
"ext_data/doctest/*",
"ext_data/doctest/invalid/*",
"ext_data/gap/*",
"ext_data/gap/joyner/*",
"ext_data/mwrank/*",
"ext_data/notebook-ipython/*",
"ext_data/nbconvert/*",
"ext_data/graphs/*",
"ext_data/pari/*",
"ext_data/pari/dokchitser/*",
"ext_data/pari/buzzard/*",
"ext_data/pari/simon/*",
"ext_data/magma/*",
"ext_data/magma/latex/*",
"ext_data/magma/sage/*",
"ext_data/valgrind/*",
"ext_data/threejs/*",
]

[tool.setuptools.dynamic]
version = {file = ["VERSION.txt"]}
Loading
Loading