Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel; remove zope_interface #29803

mkoeppe · 2020-06-05T04:52:32Z

Big upgrade steps, enabled by drop python 2 support.

https://pypi.org/project/pip/ 20.1.1
https://pypi.org/project/setuptools/ 47.1.1
https://pypi.org/project/setuptools-scm/ 4.1.2

Add new packages to satisfy dependencies:

https://pypi.org/project/wheel/ 0.34.2

These are the latest versions as of 2020-06-05; all claim to support Python >= 3.5 (note #29033)

We adjust spkg-install of several packages by removing pyproject.toml to remove the need for more extensive build infrastructure. We disable the build of zope_interface (no longer needed, see #29754).

Tarballs: See checksums.ini [upstream_url]. (To configure Sage to download from the upstream URL, use ./configure --enable-download-from-upstream-url.)

Previous updates: #26969, #27886, #26008

CC: @vbraun @fchapoton @saraedum @slel @kiwifb @dimpase @jhpalmieri

Component: packages: standard

Author: Matthias Koeppe

Branch: 371ede6

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/29803

The text was updated successfully, but these errors were encountered:

mkoeppe · 2020-06-05T05:05:29Z

Branch: u/mkoeppe/upgrade_setuptools__pip__2020_06_

mkoeppe · 2020-06-05T05:07:23Z

Author: Matthias Koeppe

mkoeppe · 2020-06-05T05:07:23Z

New commits:

`886693e`	`Upgrade pip to 20.1.1`
`80374c5`	`Upgrade setuptools to 47.1.1`
`50c6962`	`Upgrade setuptools_scm to 4.1.2`

mkoeppe · 2020-06-05T05:07:23Z

Commit: 50c6962

mkoeppe · 2020-06-05T17:26:40Z

comment:5

pip installation fails, for example on ubuntu-bionic-standard (https://github.com/mkoeppe/sage/runs/741110905)

     Preparing wheel metadata: finished with status 'error'
ERROR: Command errored out with exit status 1: /sage/local/bin/python3 /sage/local/var/tmp/sage/build/pip-20.1.1/src/src/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp32gsdf31 Check the logs for full command output.

mkoeppe · 2020-06-05T17:50:23Z

comment:6

Seems we need to add the wheel package

sagetrac-git · 2020-06-05T18:44:06Z

Changed commit from 50c6962 to 10a2bac

sagetrac-git · 2020-06-05T18:44:06Z

Branch pushed to git repo; I updated commit sha1. New commits:

`7c81b8c`	`build/pkgs/wheel: New`
`70378ed`	`build/pkgs/pip/dependencies: Add wheel, simplify $(PYTHON) python3 to $(PYTHON)`
`10a2bac`	`build/pkgs/setuptools, build/pkgs/pip: Remove python2 support from dependencies, install scripts`

mkoeppe · 2020-06-05T18:52:03Z

comment:10

Tests run at https://github.com/mkoeppe/sage/pull/36/checks

mkoeppe · 2020-06-05T21:59:48Z

comment:12

On debian-bullseye-standard:

  [pkgconfig-1.5.1]     File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
  [pkgconfig-1.5.1]       return _bootstrap._gcd_import(name[level:], package, level)
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  [pkgconfig-1.5.1]     File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
  [pkgconfig-1.5.1]   ModuleNotFoundError: No module named 'poetry'

mkoeppe · 2020-06-05T22:05:44Z

comment:13

pkgconfig is at the most current version. Looks like giving it a modern setuptools causes it to demand more.

mkoeppe · 2020-06-05T22:16:45Z

comment:15

poetry, of course, has a LONG list of additional dependencies... https://github.com/python-poetry/poetry/blob/master/pyproject.toml

mkoeppe · 2020-06-05T22:25:36Z

comment:16

We will postpone adding poetry to #29810 and instead patch pkgconfig so it installs more prosaically using setuptools.

mkoeppe · 2020-06-05T22:26:03Z

comment:17

https://github.com/matze/pkgconfig/commits/master

mkoeppe · 2020-06-05T22:48:58Z

comment:18

See #28883 for the last pkgconfig update, which commented on the issues with poetry already.

sagetrac-git · 2020-06-05T22:53:15Z

Changed commit from 10a2bac to 6755adb

mkoeppe · 2020-06-05T23:03:13Z

comment:20

Tests run again at https://github.com/mkoeppe/sage/pull/36/checks

mkoeppe · 2020-06-06T16:40:34Z

comment:21

Several failures (for example in https://github.com/mkoeppe/sage/runs/744000563):

  [entrypoints-0.3]   ModuleNotFoundError: No module named 'flit'
  [testpath-0.4.2]   ModuleNotFoundError: No module named 'flit'
  [zope_interface-4.6.0]   ImportError: cannot import name 'Feature' from 'setuptools' (/sage/local/lib/python3.7/site-packages/setuptools/__init__.py)
  [terminado-0.8.1]   ModuleNotFoundError: No module named 'flit'

mkoeppe · 2020-06-06T17:09:38Z

comment:22

Possible that zope_interface can be removed (#29754)

mkoeppe · 2020-06-06T17:13:39Z

comment:24

entrypoints is current, https://pypi.org/project/testpath/ can be upgraded to 0.4.4, https://pypi.org/project/terminado/ to 0.8.3

sagetrac-git · 2020-06-06T17:39:47Z

Branch pushed to git repo; I updated commit sha1. New commits:

`29327d3`	`build/pkgs/entrypoints: Do not provide setup.py, not needed for entrypoints 0.3`
`bcc79a5`	`build/pkgs/entrypoints/spkg-install.in: Remove pyproject.toml so that setup.py is used`
`361b98b`	`build/pkgs/terminado/spkg-install.in: Remove pyproject.toml so that setup.py is used`
`5040d7e`	`build/pkgs/testpath/spkg-install.in: Remove pyproject.toml so that setup.py is used`

sagetrac-git · 2020-06-06T17:39:47Z

Changed commit from 6755adb to 5040d7e

sagetrac-git · 2020-06-06T18:18:47Z

Changed commit from 5040d7e to 371ede6

sagetrac-git · 2020-06-06T18:18:47Z

Branch pushed to git repo; I updated commit sha1. New commits:

`371ede6`	`build/pkgs/zope_interface: Change to a dummy script package`

mkoeppe · 2020-06-06T20:14:59Z

comment:28

Builds fine on homebrew-macos-standard. Ready for review.

dimpase · 2020-06-12T18:41:29Z

Reviewer: Dima Pasechnik

dimpase · 2020-06-12T18:41:29Z

comment:29

lgtm

mkoeppe · 2020-06-12T18:55:25Z

comment:30

Thanks!

vbraun · 2020-06-27T09:26:17Z

Changed branch from u/mkoeppe/upgrade_setuptools__pip__2020_06_ to 371ede6

jhpalmieri · 2020-07-02T00:29:34Z

Changed commit from 371ede6 to none

jhpalmieri · 2020-07-02T00:29:34Z

comment:32

Not a big deal, but I'm curious: with pip, why only the change to set versions=3, instead of removing the loop for vers in $versions ... and replacing $vers with 3 in spkg-install.in?

mkoeppe · 2020-07-02T18:13:11Z

comment:33

I like to keep diffs minimal. Also minor uncertainty what the python binary will be called when python 4.0 comes out...

jhpalmieri · 2020-07-13T22:18:03Z

comment:34

I think that this ticket is leading to the following messages in the sagetex install log:

Successfully built sagetex
Installing collected packages: sagetex
  Created temporary directory: /private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-unpacked-wheel-hw7nz29q
  *** Error compiling '/private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-unpacked-wheel-hw7nz29q/sagetex-3.4.data/data/share/texmf/tex/latex/sagetex/extractsagecode.py'...
    File "/private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-unpacked-wheel-hw7nz29q/sagetex-3.4.data/data/share/texmf/tex/latex/sagetex/extractsagecode.py", line 48
      except getopt.GetoptError, err:
                               ^
  SyntaxError: invalid syntax
  
  *** Error compiling '/private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-unpacked-wheel-hw7nz29q/sagetex-3.4.data/data/share/texmf/tex/latex/sagetex/makestatic.py'...
    File "/private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-unpacked-wheel-hw7nz29q/sagetex-3.4.data/data/share/texmf/tex/latex/sagetex/makestatic.py", line 48
      except getopt.GetoptError, err:
                               ^
  SyntaxError: invalid syntax
  

Successfully installed sagetex-3.4
Removed build tracker: '/private/var/folders/cp/n8wtqs490tq5psknff1hv9qr0000gn/T/pip-req-tracker-3d0gjhof'
Removing old SageTex version(s)

real	0m3.793s
user	0m1.162s
sys	0m0.458s
Copying package files from temporary location /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-9.2.beta3/local/var/tmp/sage/build/sagetex-3.4/inst to /Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-9.2.beta3/local
Successfully installed sagetex-3.4
Deleting temporary build directory
/Users/palmieri/Desktop/Sage_stuff/sage_builds/TESTING/sage-9.2.beta3/local/var/tmp/sage/build/sagetex-3.4
Finished installing sagetex-3.4.spkg

Is it the new wheel package?

(This is on OS X using the system Python, version 3.7.7.)

mkoeppe · 2020-07-14T04:29:27Z

comment:35

pip likes to install via bdist_wheel now. Something is getting compiled that was not compiled before.

I think it reveals that the sagetex code is not compatible with python 3.

dimpase · 2020-07-14T12:12:39Z

comment:36

I am going to make a new release of sagetex soon.

jhpalmieri · 2020-07-14T16:54:15Z

comment:37

Replying to @dimpase:

I am going to make a new release of sagetex soon.

Good, that should take care of it. I'm surprised that sagetex still works, given these syntax errors.

dimpase · 2020-07-14T17:49:34Z

comment:38

probably it's from some file that we normally didn't install, or perhaps it was using some compatibility kludge that is now gone.

mkoeppe added this to the sage-9.2 milestone Jun 5, 2020

mkoeppe added c: packages: standard labels Jun 5, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Upgrade setuptools, pip (2020-06)~~ Upgrade setuptools, setuptools_scm, pip (2020-06) Jun 5, 2020

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Upgrade setuptools, setuptools_scm, pip (2020-06)~~ Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel Jun 5, 2020

This comment has been minimized.

Sign in to view

mkoeppe added the s: needs review label Jun 6, 2020

mkoeppe changed the title ~~Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel~~ Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel; remove zope_interface Jun 6, 2020

dimpase added s: positive review and removed s: needs review labels Jun 12, 2020

vbraun removed the s: positive review label Jun 27, 2020

vbraun closed this as completed in d243d25 Jun 27, 2020

ohanar mentioned this issue Aug 15, 2020

make sagelib use setuptools instead of distutils #13190

Closed

Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel; remove zope_interface #29803

Upgrade setuptools, setuptools_scm, pip (2020-06), add package wheel; remove zope_interface #29803

Comments

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

This comment has been minimized.

This comment has been minimized.

sagetrac-git mannequin commented Jun 5, 2020

sagetrac-git mannequin commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

This comment has been minimized.

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

This comment has been minimized.

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

sagetrac-git mannequin commented Jun 5, 2020

mkoeppe commented Jun 5, 2020

mkoeppe commented Jun 6, 2020

mkoeppe commented Jun 6, 2020

mkoeppe commented Jun 6, 2020

sagetrac-git mannequin commented Jun 6, 2020

sagetrac-git mannequin commented Jun 6, 2020

sagetrac-git mannequin commented Jun 6, 2020

sagetrac-git mannequin commented Jun 6, 2020

This comment has been minimized.

mkoeppe commented Jun 6, 2020

dimpase commented Jun 12, 2020

dimpase commented Jun 12, 2020

mkoeppe commented Jun 12, 2020

vbraun commented Jun 27, 2020

jhpalmieri commented Jul 2, 2020

jhpalmieri commented Jul 2, 2020

mkoeppe commented Jul 2, 2020

jhpalmieri commented Jul 13, 2020

mkoeppe commented Jul 14, 2020

dimpase commented Jul 14, 2020

jhpalmieri commented Jul 14, 2020

dimpase commented Jul 14, 2020