Reinstallation of dependencies of setuptools_scm fails with packaging-related packages #32751

vbraun · 2021-10-24T09:12:58Z

Reinstallation of a dependency of setuptools_scm fails and breaks the Python build system. Steps to reproduce:

make distclean
make setuptools_scm
sage -f packaging

Expected behavior is to rebuild, actually get

****************************************************
Uninstalling existing 'packaging'
Running pip-uninstall script for 'packaging'
Found existing installation: packaging 21.0
Uninstalling packaging-21.0:
  Successfully uninstalled packaging-21.0
Removing stamp file '/home/vbraun/Sage/git/local/var/lib/sage/installed/packaging-21.0'
Installing packaging-21.0
Processing /home/vbraun/Sage/git/local/var/tmp/sage/build/packaging-21.0/src
    Preparing wheel metadata: started
    Running command /home/vbraun/Sage/git/local/bin/python3 /home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpdfyrpf_j
    Traceback (most recent call last):
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
        main()
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 151, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 40, in <module>
        setup(
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/_distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/dist.py", line 450, in __init__
        _Distribution.__init__(
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/_distutils/dist.py", line 293, in __init__
        self.finalize_options()
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/dist.py", line 827, in finalize_options
        for ep in sorted(loaded, key=by_order):
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/setuptools/dist.py", line 826, in <lambda>
        loaded = map(lambda e: e.load(), filtered)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 2449, in load
        self.require(*args, **kwargs)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 2472, in require
        items = working_set.resolve(reqs, env, installer, extras=self.extras)
      File "/home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pkg_resources/__init__.py", line 772, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'packaging>=20.0' distribution was not found and is required by the application
    Preparing wheel metadata: finished with status 'error'
WARNING: Discarding file:///home/vbraun/Sage/git/local/var/tmp/sage/build/packaging-21.0/src. Command errored out with exit status 1: /home/vbraun/Sage/git/local/bin/python3 /home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpdfyrpf_j Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/vbraun/Sage/git/local/bin/python3 /home/vbraun/Sage/git/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpdfyrpf_j Check the logs for full command output.
************************************************************************************************************************************************************
Error building a wheel for packaging-21.0
************************************************************************************************************************************************************

Inserting some debug printing in require line 2474 reveals

self = EntryPoint.parse('setuptools_scm = setuptools_scm.integration:infer_version'), 
reqs = [Requirement.parse('setuptools'), Requirement.parse('tomli>=1.0.0'), Requirement.parse('packaging>=20.0')], None, None, ()]

This is because

cat venv/lib/python3.9/site-packages/setuptools_scm-6.3.2.dist-info/METADATA
[...]
Requires-Dist: packaging (>=20.0)
Requires-Dist: setuptools
Requires-Dist: tomli (>=1.0.0)
Provides-Extra: toml
Requires-Dist: setuptools (>=42) ; extra == 'toml'
Requires-Dist: tomli (>=1.0.0) ; extra == 'toml'
[...]

Depends on #32659

CC: @vbraun @dimpase @jhpalmieri @mkoeppe

Component: build

Author: Matthias Koeppe

Branch/Commit: 8f3bd8f

Reviewer: John Palmieri, Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/32751

The text was updated successfully, but these errors were encountered:

mkoeppe · 2021-10-24T18:01:21Z

comment:4

Thanks for investigating. I was able to reproduce this error using a shorter recipe (updated the description).

mkoeppe · 2021-10-24T18:07:34Z

Dependencies: #32659

mkoeppe · 2021-10-24T18:28:35Z

comment:6

What's happening here is simply that sage -p is broken. Don't use it.

vbraun · 2021-10-24T18:36:59Z

comment:8

I usually don't use sage -p, that was just an attempt to get a reproducer. Still get frequent incremental build failures of that sort.

mkoeppe · 2021-10-24T18:38:03Z

comment:9

Yes, that's true, there are 2 issues at play here. Let's take sage -p to #30649.

mkoeppe · 2021-10-24T18:41:29Z

comment:11

I've updated the recipe.

mkoeppe · 2021-10-24T19:09:32Z

Branch: u/mkoeppe/incremental_build_fails_with_packaging_related_packages

mkoeppe · 2021-10-24T19:10:31Z

Commit: e89e278

mkoeppe · 2021-10-24T19:10:31Z

comment:14

Here's a solution that takes us a step closer to sanity.

New commits:

`2d852ad`	`build/bin/sage-dist-helpers (sdh_pip_uninstall): Use build/bin/sage-pip-uninstall, resurrected in simplified form`
`b5a89ad`	`build/bin/sage-dist-helpers (sdh_store_and_pip_install_wheel): Uninstall before installing, to ensure the wheel is installed even if the version is the same`
`30a272e`	`build/pkgs/sage_setup/dependencies: Add interpreter specs as dependencies`
`d435781`	`build/bin/sage-dist-helpers (sdh_store_and_pip_install_wheel): Use sage-pip-uninstall`
`cc00d8c`	`Merge #32659`
`516bdb3`	`build/bin/sage-dist-helpers (sdh_pip_install): Handle new option --build-isolation`
`e89e278`	`build/pkgs/packaging: Use --build-isolation`

mkoeppe · 2021-10-24T19:11:41Z

comment:15

Probably need the same for more dependencies of setuptools_scm.

mkoeppe · 2021-10-24T19:17:06Z

Author: Matthias Koeppe

mkoeppe · 2021-10-24T19:17:06Z

comment:16

Looks like this is good enough, please try

jhpalmieri · 2021-10-27T21:17:35Z

comment:17

It looks okay to me. Volker?

vbraun · 2021-10-27T22:40:12Z

Reviewer: John Palmieri, Volker Braun

mkoeppe · 2021-10-28T00:37:03Z

comment:19

Thanks.

vbraun · 2021-10-28T22:34:04Z

comment:20

I tried a clean build (make distclean) and pillow fails to build...

vbraun · 2021-10-28T22:36:42Z

comment:21

(deleted, wrong ticket)

sagetrac-git · 2021-10-28T23:36:52Z

Changed commit from e89e278 to 8f3bd8f

sagetrac-git · 2021-10-28T23:36:52Z

Branch pushed to git repo; I updated commit sha1. New commits:

`fd91cb6`	`build/bin/sage-dist-helpers (sdh_pip_install): Pass options on to sdh_store_and_pip_install_wheel`
`8f3bd8f`	`build/bin/sage-dist-helpers: Only extract special ooptions --build-isolation and --no-deps from the front, pass --no-deps only to install`

mkoeppe · 2021-10-28T23:37:10Z

comment:23

Guessing what may have gone wrong, fixed.

jhpalmieri · 2021-10-29T02:10:37Z

comment:24

It builds for me now.

jhpalmieri · 2021-10-30T04:54:20Z

comment:25

Builds for me on OS X and with several tox docker builds. Let's try again.

mkoeppe · 2021-10-30T05:17:10Z

comment:26

Thanks!

vbraun · 2021-11-02T22:36:38Z

Changed branch from u/mkoeppe/incremental_build_fails_with_packaging_related_packages to 8f3bd8f

vbraun added this to the sage-9.5 milestone Oct 24, 2021

vbraun added c: build labels Oct 24, 2021

This comment has been minimized.

Sign in to view

mkoeppe added p: critical / 2 and removed p: major / 3 labels Oct 24, 2021

mkoeppe changed the title ~~Incremental build fails with packaging-related packages~~ Installation of packages using "sage -p" fails with packaging-related packages Oct 24, 2021

mkoeppe added p: major / 3 and removed p: critical / 2 labels Oct 24, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~Installation of packages using "sage -p" fails with packaging-related packages~~ Reinstallation of dependencies of setuptools_scm fails with packaging-related packages Oct 24, 2021

mkoeppe added p: critical / 2 and removed p: major / 3 labels Oct 24, 2021

mkoeppe added the s: needs review label Oct 24, 2021

vbraun added s: positive review and removed s: needs review labels Oct 27, 2021

vbraun added s: needs work and removed s: positive review labels Oct 28, 2021

mkoeppe added s: needs review and removed s: needs work labels Oct 28, 2021

jhpalmieri added s: positive review and removed s: needs review labels Oct 30, 2021

vbraun removed the s: positive review label Nov 2, 2021

vbraun closed this as completed in b4bb35e Nov 2, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reinstallation of dependencies of setuptools_scm fails with packaging-related packages #32751

Reinstallation of dependencies of setuptools_scm fails with packaging-related packages #32751

vbraun commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

This comment has been minimized.

vbraun commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

jhpalmieri commented Oct 27, 2021

vbraun commented Oct 27, 2021

mkoeppe commented Oct 28, 2021

vbraun commented Oct 28, 2021

vbraun commented Oct 28, 2021

sagetrac-git mannequin commented Oct 28, 2021

sagetrac-git mannequin commented Oct 28, 2021

mkoeppe commented Oct 28, 2021

jhpalmieri commented Oct 29, 2021

jhpalmieri commented Oct 30, 2021

mkoeppe commented Oct 30, 2021

vbraun commented Nov 2, 2021

Reinstallation of dependencies of setuptools_scm fails with packaging-related packages #32751

Reinstallation of dependencies of setuptools_scm fails with packaging-related packages #32751

Comments

vbraun commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

This comment has been minimized.

vbraun commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

This comment has been minimized.

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

mkoeppe commented Oct 24, 2021

jhpalmieri commented Oct 27, 2021

vbraun commented Oct 27, 2021

mkoeppe commented Oct 28, 2021

vbraun commented Oct 28, 2021

vbraun commented Oct 28, 2021

sagetrac-git mannequin commented Oct 28, 2021

sagetrac-git mannequin commented Oct 28, 2021

mkoeppe commented Oct 28, 2021

jhpalmieri commented Oct 29, 2021

jhpalmieri commented Oct 30, 2021

mkoeppe commented Oct 30, 2021

vbraun commented Nov 2, 2021