Freezing local tarball packages suggests requirement change that doesn't work

[vbabiy]

When I put a local source tarball in a requirements file and freeze the resulting environment, pip says that it doesn't know what it would install and suggests the "#egg=PackageName" syntax:

failbowl:temp2 $ echo 'REQUIREMENT FILE CONTENTS' &&
cat requirements.txt &&
rm -rf env &&
echo 'BEGIN INSTALL' &&
pip install --no-deps -E env -r requirements.txt &&
echo 'BEGIN FREEZE' &&
(. env/bin/activate && pip freeze -r requirements.txt)

REQUIREMENT FILE CONTENTS
vendor/sf.phpsession-0.1.tar.gz
BEGIN INSTALL
Creating new virtualenv environment in env
  New python executable in env/bin/python
  Installing setuptools...done.....
Unpacking ./vendor/sf.phpsession-0.1.tar.gz
  Running setup.py egg_info for package from file:///Users/grb/temp/temp2/vendor/sf.phpsession-0.1.tar.gz
Installing collected packages: sf.phpsession
  Running setup.py install for sf.phpsession
    Skipping installation of /Users/grb/temp/temp2/env/lib/python2.6/site-packages/sf/__init__.py (namespace package)
    Installing /Users/grb/temp/temp2/env/lib/python2.6/site-packages/sf.phpsession-0.1-py2.6-nspkg.pth
Successfully installed sf.phpsession
Cleaning up...
BEGIN FREEZE
Skipping line because it's not clear what it would install: vendor/sf.phpsession-0.1.tar.gz
  (add #egg=PackageName to the URL to avoid this warning)
## The following requirements were added by pip --freeze:
sf.phpsession==0.1
wsgiref==0.1.2

However, that syntax doesn't actually work for local files. The command below is the same as the previous command, but with "#egg=sf.phpsession" added to the requirement:

failbowl:temp2 $ echo 'REQUIREMENT FILE CONTENTS' &&
cat requirements.txt &&
rm -rf env &&
echo 'BEGIN INSTALL' &&
pip install --no-deps -E env -r requirements.txt
&& echo 'BEGIN FREEZE'
&& (. env/bin/activate && pip freeze -r requirements.txt)

REQUIREMENT FILE CONTENTS
vendor/sf.phpsession-0.1.tar.gz#egg=sf.phpsession
BEGIN INSTALL
Creating new virtualenv environment in env
  New python executable in env/bin/python
  Installing setuptools...done.....
Exception:
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/basecommand.py", line 120, in main
    self.run(options, args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/commands/install.py", line 158, in run
    for req in parse_requirements(filename, finder=finder, options=options):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 1395, in parse_requirements
    req = InstallRequirement.from_line(line, comes_from)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 98, in from_line
    return cls(req, comes_from, url=url)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 38, in __init__
    req = pkg_resources.Requirement.parse(req)
  File "/Users/grb/temp/temp2/env/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 2510, in parse
    reqs = list(parse_requirements(s))
  File "/Users/grb/temp/temp2/env/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 2436, in parse_requirements
    line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec")
  File "/Users/grb/temp/temp2/env/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 2404, in scan_list
    raise ValueError("Expected "+item_name+" in",line,"at",line[p:])
ValueError: ('Expected version spec in', 'vendor/sf.phpsession-0.1.tar.gz#egg=sf.phpsession', 'at', '/sf.phpsession-0.1.tar.gz#egg=sf.phpsession')

Storing complete log in /Users/grb/.pip/pip.log

I'm not sure whether this should be considered a bug in the suggestion message or a bug in the requirement parsing code (or, equally possible, a bug in my understanding of what I'm making pip do. :). It seems like I should be able to add "#egg=" to a local package install, so I'm leaning toward "a bug in the requirement parsing code."

---

• Bitbucket: https://bitbucket.org/ianb/pip/issue/129
• Originally Reported By: Gary Bernhardt
• Originally Created At: 2010-05-28 00:51:03

[Ignas]

This still happens, at least to me with pip 1.1 on python 2.7

[dstufft]

The original bug report is a bit difficult to decipher, but the underlying issue here is that when freezing with a requirements file that references a file path, the output suggests syntax that does not actually work. The suggested syntax should be valid syntax.

[pradyunsg]

@dstufft What's the right way to refer to a local file (say ./dist/pkg-1.0.0.tar.gz)?

[chrahunt]

This is still present in the latest pip. Cleaner example and output:

t.sh

#!/bin/sh
cd "$(mktemp -d)"

python -m venv env
PYTHON="$PWD/env/bin/python"
$PYTHON -m pip install --upgrade pip

$PYTHON -V

cat <<EOF > setup.py
from setuptools import setup

setup(
    name='hello',
    version='0.1.0',
)
EOF

echo "= Building dist ======================================"

$PYTHON setup.py sdist

echo "= Installing dist ===================================="
echo "./dist/hello-0.1.0.tar.gz" > requirements.txt
$PYTHON -m pip install -r requirements.txt

echo "= Running freeze ====================================="
$PYTHON -m pip freeze -r requirements.txt

echo "./dist/hello-0.1.0.tar.gz#egg=hello" > requirements.txt
echo "= Installing dist 2 =================================="
$PYTHON -m pip install -r requirements.txt

echo "= Running freeze 2 ==================================="
$PYTHON -m pip freeze -r requirements.txt

echo "hello @ ./dist/hello-0.1.0.tar.gz" > requirements.txt
echo "= Installing dist 3 =================================="
$PYTHON -m pip install -r requirements.txt

echo "= Running freeze 3 ==================================="
$PYTHON -m pip freeze -r requirements.txt

Output

Collecting pip
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-19.3.1
Python 3.8.0
= Building dist ======================================
running sdist
running egg_info
creating hello.egg-info
writing hello.egg-info/PKG-INFO
writing dependency_links to hello.egg-info/dependency_links.txt
writing top-level names to hello.egg-info/top_level.txt
writing manifest file 'hello.egg-info/SOURCES.txt'
reading manifest file 'hello.egg-info/SOURCES.txt'
writing manifest file 'hello.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating hello-0.1.0
creating hello-0.1.0/hello.egg-info
copying files to hello-0.1.0...
copying setup.py -> hello-0.1.0
copying hello.egg-info/PKG-INFO -> hello-0.1.0/hello.egg-info
copying hello.egg-info/SOURCES.txt -> hello-0.1.0/hello.egg-info
copying hello.egg-info/dependency_links.txt -> hello-0.1.0/hello.egg-info
copying hello.egg-info/top_level.txt -> hello-0.1.0/hello.egg-info
Writing hello-0.1.0/setup.cfg
creating dist
Creating tar archive
removing 'hello-0.1.0' (and everything under it)
= Installing dist ====================================
Processing ./dist/hello-0.1.0.tar.gz
Installing collected packages: hello
  Found existing installation: hello 0.1.0
    Not uninstalling hello at /tmp/user/1000/tmp.v7GEedEP9H, outside environment /tmp/user/1000/tmp.v7GEedEP9H/env
    Can't uninstall 'hello'. No files were found to uninstall.
    Running setup.py install for hello ... done
Successfully installed hello-0.1.0
= Running freeze =====================================
Skipping line in requirement file [requirements.txt] because it's not clear what it would install: ./dist/hello-0.1.0.tar.gz
  (add #egg=PackageName to the URL to avoid this warning)
## The following requirements were added by pip freeze:
hello==0.1.0
= Installing dist 2 ==================================
ERROR: Invalid requirement: './dist/hello-0.1.0.tar.gz#egg=hello' (from line 1 of requirements.txt)
Hint: It looks like a path. File './dist/hello-0.1.0.tar.gz#egg=hello' does not exist.
= Running freeze 2 ===================================
ERROR: Invalid requirement: './dist/hello-0.1.0.tar.gz#egg=hello'
Hint: It looks like a path. File './dist/hello-0.1.0.tar.gz#egg=hello' does not exist.
= Installing dist 3 ==================================
ERROR: Invalid requirement: 'hello @ ./dist/hello-0.1.0.tar.gz' (from line 1 of requirements.txt)
Hint: It looks like a path. File 'hello @ ./dist/hello-0.1.0.tar.gz' does not exist.
= Running freeze 3 ===================================
ERROR: Invalid requirement: 'hello @ ./dist/hello-0.1.0.tar.gz'
Hint: It looks like a path. File 'hello @ ./dist/hello-0.1.0.tar.gz' does not exist.

Not sure what the best course of action here is. Suggesting direct reference syntax would be best, but it doesn't work for relative paths. Maybe we consider this blocked on #6658?

[sbidoul]

This relates to #609 (pip freeze). Relative path direct references are also discussed in this thread, via #6658.

The direction this takes is currently as follow:

• relative paths are valid top level requirements (in requirements.txt or as command line arguments), so ./dist/hello-0.1.0.tar.gz is valid
• ./dist/hello-0.1.0.tar.gz#egg=hello is invalid
• hello @ file:///absolute/path/dist/hello-0.1.0.tar.gz is valid
• hello @ ./dist/hello-0.1.0.tar.gz is invalid (rationale), at least in Install Requires metadata
• pip freeze emits PEP 440 requirements with absolute paths: hello @ file:///absolute/path/hello-0.1.0.tar.gz (rationale) (Better freeze of distributions installed from direct URL references #7612)

[gutsytechster]

I executed the script defined in #192 (comment) with the latest master and with --unstable-feature=resolver. However, it fails. Maybe because of the specifications described in #192 (comment).

What should be the approach for solving this issue?

[DiddiLeija]

Hello! Does this issue persist? If not, can we close this issue?

[Ignas]

Ran the test script from #192 (comment) and still can repro:

Collecting pip
  Using cached https://files.pythonhosted.org/packages/47/ef/8b5470b5b94b36231ed9c0bde90caa71c0d4322d4a15f009b2b7f4287fe0/pip-22.3-py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
Successfully installed pip-22.3
Python 3.8.2
= Building dist ======================================
running sdist
running egg_info
creating hello.egg-info
writing hello.egg-info/PKG-INFO
writing dependency_links to hello.egg-info/dependency_links.txt
writing top-level names to hello.egg-info/top_level.txt
writing manifest file 'hello.egg-info/SOURCES.txt'
reading manifest file 'hello.egg-info/SOURCES.txt'
writing manifest file 'hello.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

creating hello-0.1.0
creating hello-0.1.0/hello.egg-info
copying files to hello-0.1.0...
copying setup.py -> hello-0.1.0
copying hello.egg-info/PKG-INFO -> hello-0.1.0/hello.egg-info
copying hello.egg-info/SOURCES.txt -> hello-0.1.0/hello.egg-info
copying hello.egg-info/dependency_links.txt -> hello-0.1.0/hello.egg-info
copying hello.egg-info/top_level.txt -> hello-0.1.0/hello.egg-info
Writing hello-0.1.0/setup.cfg
creating dist
Creating tar archive
removing 'hello-0.1.0' (and everything under it)
= Installing dist ====================================
Processing ./dist/hello-0.1.0.tar.gz
  Preparing metadata (setup.py) ... done
Installing collected packages: hello
  DEPRECATION: hello is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for hello ... done
Successfully installed hello-0.1.0
= Running freeze =====================================
Skipping line in requirement file [requirements.txt] because it's not clear what it would install: ./dist/hello-0.1.0.tar.gz
  (add #egg=PackageName to the URL to avoid this warning)
## The following requirements were added by pip freeze:
hello==0.1.0
= Installing dist 2 ==================================
ERROR: Invalid requirement: './dist/hello-0.1.0.tar.gz#egg=hello' (from line 1 of requirements.txt)
Hint: It looks like a path. File './dist/hello-0.1.0.tar.gz#egg=hello' does not exist.
= Running freeze 2 ===================================
ERROR: Invalid requirement: './dist/hello-0.1.0.tar.gz#egg=hello'
Hint: It looks like a path. File './dist/hello-0.1.0.tar.gz#egg=hello' does not exist.
= Installing dist 3 ==================================
ERROR: Invalid requirement: 'hello @ ./dist/hello-0.1.0.tar.gz' (from line 1 of requirements.txt)
Hint: It looks like a path. File 'hello @ ./dist/hello-0.1.0.tar.gz' does not exist.
= Running freeze 3 ===================================
ERROR: Invalid requirement: 'hello @ ./dist/hello-0.1.0.tar.gz'
Hint: It looks like a path. File 'hello @ ./dist/hello-0.1.0.tar.gz' does not exist.

[uranusjr]

The suggestion should be changed to use the PEP 508 form.