sagelib: Update metadata for PyPI deployment as package sagemath-standard

[mkoeppe]

We update the following items in build/pkgs/sagelib/src/:

• Add README.rst
• Add LICENSE.txt (for sagelib and documentation only - see discussion in Install COPYING.txt #21571)
• setup.py metadata (author, license, compatibility, ....)

We switch from distutils to setuptools so we can put the new metadata in the setup.cfg format.

sage-devel post regarding naming:

• https://groups.google.com/g/sage-devel/c/3mCIV9LnP9c/m/lQXLEGKcCQAJ

Naming was discussed during https://wiki.sagemath.org/days111 and a decision made to use the sagemath-... format and in particular sagemath-standard.

CC: @dimpase @jhpalmieri @mmasdeu @videlec @malb @kiwifb @embray @NathanDunfield @slel @orlitzky @fchapoton @vbraun @seblabbe @williamstein @mwageringel @saraedum

Component: build

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: 7ad4c0e

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/30912

[mkoeppe]

comment:1

Currently, setup.py thinks that the name of the distribution is sage.

• sage is taken - see Claim "sage" package on PyPI, replace unexplained package "sage 0.0.0" #28796 Claim "sage" package on PyPI, replace unexplained package "sage 0.0.0"
• sagemath is taken - https://pypi.org/project/sagemath/ "This is a dummy package that checks that Sage is installed in the system"
• various other related and unrelated packages are published as sage-* - https://pypi.org/search/?q=sage

I was planning to change it to sagelib. But we could also try a systematic approach regarding branding and call it sagemath-standard (in anticipation of the modularization project's smaller distributions that could be called sagemath-core, sagemath-ntl, etc., see #29705).

[mkoeppe]

Branch: u/mkoeppe/sagelib__update_metadata_for_pypi_deployment

[mkoeppe]

New commits:

2c40179

build/pkgs/sagelib/src: Update metadata

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

Commit: 2c40179

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2a91862

build/pkgs/sagelib/src/README.rst: Add some text

[sagetrac-git]

Changed commit from 2c40179 to 2a91862

[dimpase]

comment:5

should we also claim https://pypi.org/project/sagemath/ ?

[mmasdeu]

comment:7

The dummy package on pypi.org/project/sagemath was created so that other projects could be made to depend on it, and thus allow a user to install a custom package without having to recompile Sage. This has worked well to this day, you can install the darmonpoints package with a one-liner and alongside it checks that the installed version of Sage is the correct one.

At the time (2016 maybe?) we thought that this was a good idea, since it seemed like it was a long time until modularization would be a reality.

The dummy package only has one function (check_version) which compares the version passed as parameter with the sage version (via from sage.all import version).

I don't mind giving away this package if modularity is happening, since then there would a proper way to check for each of the required packages (there should be a way to check for the "whole sage" being installed, since it may be hard to decide on all the dependencies).

How should we go about it? I would rather have this kind of functionality (and I am assuming that the other two package maintainers that use the dummy package would be with me) available during the transition, specially if it takes more than a week or two...

[mkoeppe]

comment:8

I certainly don't need the sagemath name itself any time soon - I would use suffixed names such as sagemath-standard.

[tobiasdiez]

comment:13

What's the reason that this new metadata is only in build/.../sagelib and not (also?) under src?
Other than that, the changes look good to me.

Moreover, FYI, PEP 621 for Storing project metadata in pyproject.toml has been provisionally accepted (and will be fully accepted latest on 2021-03-01). Once setuptools supports this, I would propose to follow it and put the metadata in pyproject.toml.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

6432727	Merge tag '9.3.beta2' into t/30912/sagelib__update_metadata_for_pypi_deployment
4a693f2	Move build/pkgs/sagelib/src/setup.cfg to SAGE_ROOT/src, replace by symlink
ea182d7	Copy changes from build/pkgs/sagelib/src to src

[sagetrac-git]

Changed commit from 2a91862 to ea182d7

[mkoeppe]

comment:15

Replying to @tobiasdiez:

What's the reason that this new metadata is only in build/.../sagelib and not (also?) under src?

I guess I had wanted to avoid merge conflicts with #30371, but here you go.

[mkoeppe]

comment:16

Replying to @tobiasdiez:

Moreover, FYI, PEP 621 for Storing project metadata in pyproject.toml has been provisionally accepted (and will be fully accepted latest on 2021-03-01). Once setuptools supports this, I would propose to follow it and put the metadata in pyproject.toml.

Yes, it's good to see the progress in this direction.

[sagetrac-git]

Changed commit from ea182d7 to a1a10b9

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a1a10b9

src/VERSION.txt: New

[mkoeppe]

Changed keywords from none to sd111

[slel]

comment:23

Replying to @mmasdeu:

The dummy package on pypi.org/project/sagemath was created so that
other projects could be made to depend on it, and thus allow a user
to install a custom package without having to recompile Sage.
This has worked well to this day, you can install the darmonpoints
package with a one-liner and alongside it checks that the installed
version of Sage is the correct one.

At the time (2016 maybe?) we thought that this was a good idea,
since it seemed like it was a long time until modularization
would be a reality.

The dummy package only has one function (check_version) which
compares the version passed as parameter with the sage version
(via from sage.all import version).

I don't mind giving away this package if modularity is happening,
since then there would a proper way to check for each of the required
packages (there should be a way to check for the "whole sage" being
installed, since it may be hard to decide on all the dependencies).

How should we go about it? I would rather have this kind of
functionality (and I am assuming that the other two package
maintainers that use the dummy package would be with me) available
during the transition, specially if it takes more than a week or two...

Maybe there could be a package with a more
descriptive name for that use case?

Maybe one of the following names, or something similar?

• sagemath_installed
• sagemath_version
• sagemath_checkinstalled
• sagemath_checkversion
• sagemath_probe

Some users assume from the name that it actually installs Sage,
and are surprised after installing it that things don't work as
expected; this comes up once in a while on the help and support
channels such as Ask Sage, sage-support, Stack Overflow...

Not an emergency, but something to consider.

[mkoeppe]

Work Issues: add license file

[sagetrac-git]

Changed commit from deb9eb3 to 7ad4c0e

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

7ad4c0e

Merge tag '9.3.beta4' into t/30912/sagelib__update_metadata_for_pypi_deployment

[mkoeppe]

comment:29

Still hoping for someone with recent Python packaging experience to review this ticket.

[mkoeppe]

comment:34

Needs review.

[mkoeppe]

comment:35

Can we please get this into 9.3?

[jhpalmieri]

comment:36

I'm seeing a doctest failure:

sage -t --warn-long 95.5 --random-seed=0 src/sage/all.py
**********************************************************************
File "src/sage/all.py", line 333, in sage.all._write_started_file
Failed example:
    os.path.isfile(started_file)  # optional - build
Expected:
    True
Got:
    False
**********************************************************************

Is it spurious or could it be caused by this ticket?

[mkoeppe]

comment:37

If the docbuild fails with an error, then make does not get to the target that creates the started_file. You can fix this with make build-start

[jhpalmieri]

comment:38

That's too bad. I thought, but maybe I'm wrong about this, that starting Sage used to create the started_file, and I made sure to run Sage before rerunning this doctest. You're right, in any case: make build-start fixed the doctest.

[mkoeppe]

comment:39

I don't think starting sage creates the started file. It's only happening as part of the build in build/bin/sage-starts

[mkoeppe]

comment:40

since #13988

[jhpalmieri]

Reviewer: John Palmieri

[jhpalmieri]

comment:41

I am far from an expert in setuptools, but this looks okay to me.

[mkoeppe]

comment:42

Thanks!

[kiwifb]

comment:43

This is fun. Volker is in the process of merging the ticket, which means I get to test it in sage-on-gentoo

>>> Compiling source in /dev/shm/portage/sci-mathematics/sage-9999/work/sage-9999/src ...
 * python3_7: running distutils-r1_run_phase python_compile
python3.7 setup.py build -j 8
distributions = ['', 'sage-bliss', 'sage-meataxe']
Discovering Python/Cython source code....
Discovered Python/Cython sources, time: 0.20 seconds.
************************************************************************
Traceback (most recent call last):
  File "setup.py", line 153, in <module>
    ext_modules = cython_modules)
  File "/usr/lib/python3.7/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 121, in setup
    dist.parse_config_files()
  File "/usr/lib/python3.7/site-packages/setuptools/dist.py", line 668, in parse_config_files
    ignore_option_errors=ignore_option_errors)
  File "/usr/lib/python3.7/site-packages/setuptools/config.py", line 157, in parse_configuration
    meta.parse()
  File "/usr/lib/python3.7/site-packages/setuptools/config.py", line 463, in parse
    section_parser_method(section_options)
  File "/usr/lib/python3.7/site-packages/setuptools/config.py", line 436, in parse_section
    self[name] = value
  File "/usr/lib/python3.7/site-packages/setuptools/config.py", line 220, in __setitem__
    value = parser(value)
  File "/usr/lib/python3.7/site-packages/setuptools/config.py", line 548, in _parse_version
    raise DistutilsOptionError(tmpl.format(**locals()))
distutils.errors.DistutilsOptionError: Version loaded from file: VERSION.txt does not comply with PEP 440: SageMath version 9.3.beta7, Release Date: 2021-02-07
************************************************************************
Error building the Sage library

You can read the PEP in question here https://www.python.org/dev/peps/pep-0440/. I am currently using the Gentoo stable setuptools which is 50.3.0. sage is shipping 51.1.1 so I am guessing someone had a change of heart about strict enforcement between those two. I'll check by updating my own to the latest in Gentoo's tree (53.0.0).

But if the PEP get enforced the days of naming release .betaN and .rcN are numbered.

[kiwifb]

comment:44

Reading more closely a, b and rc are allowed.

[kiwifb]

comment:45

Still broken for me with setuptools-53.0.0.

[mkoeppe]

comment:46

This does not affect the Sage distribution (it uses the other copy of setup.py) and is fixed already in #31357.

[kiwifb]

comment:47

I see. Switching to that copy is possible but that messes up how I currently do the documentation. I am not quite sure how much effort yet but it probably should be done.

[kiwifb]

comment:48

Replying to @kiwifb:

I see. Switching to that copy is possible but that messes up how I currently do the documentation. I am not quite sure how much effort yet but it probably should be done.

Lesson of the day, patch doesn't like symbolic links which means that I am better off waiting for #31357.

[vbraun]

Changed branch from u/mkoeppe/sagelib__update_metadata_for_pypi_deployment to 7ad4c0e