pip-19.x: pip wheel does not pick a whl on PEP-517 project

[gjo]

Environment

• pip version: 19.0, 19.0.1
• Python version: 3.6.6, 3.4.3
• OS: ubuntu, macosx

Description
It seems that pep517's build_wheel passes existing distfiles.
However, pip uses os.listdir()'s first item.
https://github.com/pypa/pip/blob/master/src/pip/_internal/wheel.py#L751

When os.listdir(temp_dir.path) returns ['somepkg-1.0.tar.bz2', 'somepkg-1.0-py2.py3-none-any.whl'], somepkg-1.0.tar.bz2 is copied to wheelhouse directory.

How to Reproduce

1. create a project with pyproject.toml
2. run python setup.py sdist
3. run pip wheel -f wheelhouse -w wheelhouse .

Workarounds

• clean dist directory before run pip wheel
• use pip wheel --no-use-pep517
• use pip-18.1

[gjo]

a similar case is reproduced in pip install:

1. create a project with pyproject.toml
2. run python setup.py sdist
3. create venv and cd to it. (python3 -m venv /path/to/env && cd /path/to/env && bin/pip install -U pip)
4. bin/pip install /path/to/project

Stacktrace

Exception:
Traceback (most recent call last):
  File "/PATH/TO/VENV/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/PATH/TO/VENV/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "//PATH/TO/VENV/lib/python3.6/site-packages/pip/_internal/wheel.py", line 941, in build
    assert req.link.is_wheel
AssertionError

[pradyunsg]

Hi @gjo! Thanks for filing this!

$ python3 -m venv /tmp/pip-6196-tester
$ cd /tmp/pip-6196-tester
$ bin/pip install -U pip
$ mkpkg trial
$ touch trial/pyproject.toml
$ cd trial
$ python setup.py sdist
$ ../bin/pip wheel -f wheelhouse -w wheelhouse .
Looking in links: wheelhouse
Processing /private/tmp/pip-6196-tester/trial
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: trial
  Building wheel for trial (PEP 517) ... done
  Stored in directory: /private/tmp/pip-6196-tester/trial/wheelhouse
Successfully built trial
$ tree wheelhouse
wheelhouse
└── trial-0.1.0.tar.gz

0 directories, 1 file

So... I can reproduce this issue! Thanks for a detailed report! :)

---

The install case seems to be working fine for me. I tried this locally:

$ python3 -m venv /tmp/pip-6196-tester
$ cd /tmp/pip-6196-tester
$ bin/pip install -U pip --quiet
$ mkpkg trial  # https://github.com/pradyunsg/dotfiles/blob/master/src/python/aliases.zsh
$ touch trial/pyproject.toml
$ tree trial
trial
├── setup.py
├── pyproject.toml
└── trial
    └── __init__.py

1 directory, 3 files
$ bin/pip install ./trial
Processing ./trial
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: trial
  Building wheel for trial (PEP 517) ... done
  Stored in directory: /private/var/folders/4d/bt0_xfx56bjfmmt2bv3r5_qh0000gn/T/pip-ephem-wheel-cache-qo7imeut/wheels/d9/26/8a/63a8ca08ceb31ad86bf93bd6f9913bf64a86363730b6359887
Successfully built trial
Installing collected packages: trial
Successfully installed trial-0.1.0

[pradyunsg]

Okay, so PEP 517 backends return a basename. I'll make changes so that pip will use that backend name itself, when using PEP 517 to building wheels using it.

[pradyunsg]

And... I have trouble reproducing this in the test suite. Turns out os.listdir order is arbitrary and thus this issue doesn't surface in a deterministic manner for me. Making a PR without the test for this until then.

@cjerdonek @pfmoore Any suggestions for how to test this?

My latest attempt at writing the test

def test_pep517_wheels_are_not_confused_with_other_files(script, tmpdir, data):
    """Check correct wheels are copied. (#6196)
    """
    pkg_to_wheel = data.src / 'withpyproject'
    (pkg_to_wheel / 'dist').makedirs()
    (pkg_to_wheel / 'dist' / 'withpyproject-0.0.1.tar.gz').write("hello")

    result = script.pip('wheel', pkg_to_wheel, '-w', script.scratch_path)
    assert "Installing build dependencies" in result.stdout, result.stdout

    wheel_file_name = 'withpyproject-0.0.1-py%s-none-any.whl' % pyversion[0]
    wheel_file_path = script.scratch_path / wheel_file_name
    assert wheel_file_path in result.files_created, result.stdout

[cjerdonek]

For starters, I would remove the indeterminism from pip’s code base by making it sorted(os.listdir(...)) and then go from there.

[pradyunsg]

Cool. I'll do that then. :)

[gjo]

Thank you @pradyunsg !

pip install's case can reproduce with this procedure:

$ mkdir /PATH/TO/testing
$ cd /PATH/TO/testing
$ mkdir -p proj1/proj1
$ cat > proj1/setup.py <<EOF
from setuptools import setup
setup()
EOF
$ cat > proj1/setup.cfg <<EOF
[metadata]
name = proj1
version = 0.1

[options]
packages = find:
EOF
$ touch proj1/pyproject.toml
$ python3 -m venv env
$ env/bin/pip install -U pip
$ cd proj1
$ ../env/bin/python setup.py sdist
$ ../env/bin/pip install .

[pradyunsg]

Thanks for filing this @gjo and the clear reproduction instructions! :)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.