pip install wheel vs python -m pip install wheel differing compliance for PEP 508

[cam72cam]

Environment

• pip version: 531be4f up through master
• Python version: python=3.5.5
• OS: RHEL 7.5

Description
Package with "package@http://remote/package.whl" in install_requires only installs correctly with python -m pip install ${result_of_bdist_wheel}

Running pip install ${result_of_bdist_wheel} exits with an error:

Collecting ${dependency} (from ${result_of_bdist_wheel}==0.1)
  1 location(s) to search for versions of ${dependency}:
  * https://pypi.org/simple/${dependency}/
  Getting page https://pypi.org/simple/${dependency}/
  Looking up "https://pypi.org/simple/${dependency}/" in the cache
  No cache entry available
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/${dependency}/ HTTP/1.1" 404 13
  Status code 404 not in (200, 203, 300, 301)
  Could not fetch URL https://pypi.org/simple/${dependency}/: 404 Client Error: Not Found for url: https://pypi.org/simple/${dependency}/ - skipping
  Could not find a version that satisfies the requirement ${dependency} (from ${result_of_bdist_wheel}==0.1) (from versions: )
Cleaning up...

Expected behavior

Installing this wheel should have the same dependency lookup with python -m pip install vs pip install

How to Reproduce

pip install git+https://github.com/pypa/pip.git@531be4f8863b96e168e8780aeea025b706658268
or
pip install git+https://github.com/pypa/pip.git

from setuptools import setup

setup(
    name='package',
    version='0.1',
    description='package',
    url='',
    author='',
    author_email='[email protected]',
    license='',
    packages=['package'],
    zip_safe=False,
    install_requires=[
        "dependency@https://remote.url/path/to/dependency-0.1-py3-none-any.whl",
    ],
)

[benoit-pierre]

The difference in behaviour between python -m pip install and pip install does not make any sense. Are you sure you're not hitting #5736?

[benoit-pierre]

Please check python -m pip install and pip install use the same pip.

[cam72cam]

(domain-mast) cmesh@peppercorn:/internal/1/ASB $ pip --version
pip 18.1.dev0 from /internal/1/ASB/conda/envs/domain-mast/lib/python3.5/site-packages/pip (python 3.5)
(domain-mast) cmesh@peppercorn:/internal/1/ASB $ python -m pip --version
pip 18.1.dev0 from /internal/1/ASB/conda/envs/domain-mast/lib/python3.5/site-packages/pip (python 3.5)

[benoit-pierre]

I still don't see how you could get different behaviours... Can you past the complete output of running both commands with -v?

I would expect both invocations to fail because of pypa/wheel#249, since the generated wheel metadata is going to be invalid. You can check with:

> unzip -p dist/package-0.1-py3-none-any.whl '*/METADATA'                                        
Metadata-Version: 2.1
Name: package
Version: 0.1
Summary: package
Home-page: UNKNOWN
Author: UNKNOWN
Author-email: [email protected]
License: UNKNOWN
Platform: UNKNOWN
Requires-Dist: dependency

UNKNOWN

As you can see, the direct URL part of the requirement has been stripped.

Note: there's a dependency project on PyPI, so not a great choice for test data.

[cam72cam]

hmm, interesting. The deps are definitely missing

Metadata-Version: 2.1
Name: domain.mast
Version: 0.1
Summary: MAST domain-mast
Home-page: https://domain-mast.stsci.edu
Author: MAST Developers
Author-email: [email protected]
License: UNKNOWN
Platform: UNKNOWN
Requires-Dist: asb.core
Requires-Dist: asb.config

UNKNOWN

Here's the output:
https://gist.github.com/cam72cam/f7c12417226a26b673621db53b75d171

The strangest part is that if I cd into the parent directory and run the 'python -m pip command it fails as expected'

I can try out your PR and see if it generates the correct manifest

[cam72cam]

If I install your wheel PR, it seems to fix the wheel.

So at this point the ticket is highlighting a difference where python -m pip install package.whl may be reading the setup.py from the current directory?

[benoit-pierre]

Yeah, it's because the current directory gets added to sys.path when using python -m pip:

Requirement already satisfied: domain.mast==0.1 from file:///internal/1/ASB/mast-services-template/domain-mast/dist/domain.mast-0.1-py3-none-any.whl in /internal/1/ASB/mast-services-template/domain-mast (0.1)

[benoit-pierre]

It does not read setup.py, but because the current directory is added to sys.path and the egg-info metadata is present, the package, from the point of view of pip, is already available. And the egg-info metadata (requires.txt) is valid, unlike the wheel metadata.

[benoit-pierre]

So it is a duplicate of #5736, and will be fixed once pypa/wheel#249 is taken care of. Thanks for testing by the way!

[cam72cam]

Ah, that makes sense. Thanks for pushing for PEP508 support, it's making my life much easier 👍

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.