pip 20.3b1 fails to reinstall packages in -e . mode (despite changes)

[bluenote10]

Environment

• pip version: 20.3b1
• Python version: Python 3.7.9
• OS: Ubuntu 16.04

Description

Starting with version 20.3b1, pip install -e . fails to re-install a package even though it has changes that should lead to a re-installation.

Expected behavior

Automatic re-installation was the standard in earlier pip versions and may be the preferred/expected behavior?

How to Reproduce

Consider this minimal example structure:

setup.py

from setuptools import setup, find_packages
setup(
    name='foobar',
    packages=find_packages(),
)

setup.cfg

[options.entry_points]
console_scripts =
    foo1 = foobar.foo1:main

foobar/foo1.py

def main():
    print("Hello world")

Now running pip install -e . -vvv for the first time leads to:

stdout

Using pip 20.3b1 from /home/user/debug/pip_issue/tmp_venv/lib/python3.7/site-packages/pip (python 3.7)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-kqycj3y7
Created temporary directory: /tmp/pip-req-tracker-e34ev8zn
Initialized build tracking at /tmp/pip-req-tracker-e34ev8zn
Created build tracker: /tmp/pip-req-tracker-e34ev8zn
Entered build tracker: /tmp/pip-req-tracker-e34ev8zn
Created temporary directory: /tmp/pip-install-e2piun_j
Obtaining file:///home/user/debug/pip_issue
  Added file:///home/user/debug/pip_issue to build tracker '/tmp/pip-req-tracker-e34ev8zn'
    Running setup.py (path:/home/user/debug/pip_issue/setup.py) egg_info for package from file:///home/user/debug/pip_issue
    Created temporary directory: /tmp/pip-pip-egg-info-3xi_vlpj
    Running command python setup.py egg_info
    running egg_info
    creating /tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info
    writing /tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/dependency_links.txt
    writing entry points to /tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/entry_points.txt
    writing top-level names to /tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/top_level.txt
    writing manifest file '/tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/SOURCES.txt'
    reading manifest file '/tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/SOURCES.txt'
    writing manifest file '/tmp/pip-pip-egg-info-3xi_vlpj/foobar.egg-info/SOURCES.txt'
  Source in /home/user/debug/pip_issue has version 0.0.0, which satisfies requirement foobar==0.0.0 from file:///home/user/debug/pip_issue
  Removed foobar==0.0.0 from file:///home/user/debug/pip_issue from build tracker '/tmp/pip-req-tracker-e34ev8zn'
Created temporary directory: /tmp/pip-unpack-67rn1ufq
Installing collected packages: foobar
  Running setup.py develop for foobar
    Running command /home/user/debug/pip_issue/tmp_venv/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/user/debug/pip_issue/setup.py'"'"'; __file__='"'"'/home/user/debug/pip_issue/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
    running develop
    running egg_info
    creating foobar.egg-info
    writing foobar.egg-info/PKG-INFO
    writing dependency_links to foobar.egg-info/dependency_links.txt
    writing entry points to foobar.egg-info/entry_points.txt
    writing top-level names to foobar.egg-info/top_level.txt
    writing manifest file 'foobar.egg-info/SOURCES.txt'
    reading manifest file 'foobar.egg-info/SOURCES.txt'
    writing manifest file 'foobar.egg-info/SOURCES.txt'
    running build_ext
    Creating /home/user/debug/pip_issue/tmp_venv/lib/python3.7/site-packages/foobar.egg-link (link to .)
    Adding foobar 0.0.0 to easy-install.pth file
    Installing foo1 script to /home/user/debug/pip_issue/tmp_venv/bin

    Installed /home/user/debug/pip_issue
Successfully installed foobar
Removed build tracker: '/tmp/pip-req-tracker-e34ev8zn'

The entrypoint foo1 is available at that point. So far so good.

Now duplicating foobar/foo1.py into foobar/foo2.py, adding the corresponding entry point foo2 = foobar.foo2:main, and rerunning pip install -e . -vvv results in:

stdout

Using pip 20.3b1 from /home/user/debug/pip_issue/tmp_venv/lib/python3.7/site-packages/pip (python 3.7)
Non-user install because user site-packages disabled
Created temporary directory: /tmp/pip-ephem-wheel-cache-uijdj_b0
Created temporary directory: /tmp/pip-req-tracker-afztj7ue
Initialized build tracking at /tmp/pip-req-tracker-afztj7ue
Created build tracker: /tmp/pip-req-tracker-afztj7ue
Entered build tracker: /tmp/pip-req-tracker-afztj7ue
Created temporary directory: /tmp/pip-install-qgiftod1
Obtaining file:///home/user/debug/pip_issue
  Added file:///home/user/debug/pip_issue to build tracker '/tmp/pip-req-tracker-afztj7ue'
    Running setup.py (path:/home/user/debug/pip_issue/setup.py) egg_info for package from file:///home/user/debug/pip_issue
    Created temporary directory: /tmp/pip-pip-egg-info-p6ue4r4c
    Running command python setup.py egg_info
    running egg_info
    creating /tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info
    writing /tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/dependency_links.txt
    writing entry points to /tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/entry_points.txt
    writing top-level names to /tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/top_level.txt
    writing manifest file '/tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/SOURCES.txt'
    reading manifest file '/tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/SOURCES.txt'
    writing manifest file '/tmp/pip-pip-egg-info-p6ue4r4c/foobar.egg-info/SOURCES.txt'
  Source in /home/user/debug/pip_issue has version 0.0.0, which satisfies requirement foobar==0.0.0 from file:///home/user/debug/pip_issue
  Removed foobar==0.0.0 from file:///home/user/debug/pip_issue from build tracker '/tmp/pip-req-tracker-afztj7ue'
Created temporary directory: /tmp/pip-unpack-o99n9581
Removed build tracker: '/tmp/pip-req-tracker-afztj7ue'

As a user I would expect entrypoint foo2 to be available now as well, but it isn't.

In previous pip versions (verified against 20.2.4) this kind of pip install -e . workflow for local packages without explicit versioning was working fine.

The obvious work-around is to use --force, but the question is if skipping installs is really intended.

Output

See inline above.

[ichard26]

Maybe related to #8711?

[bluenote10]

Maybe related to #8711?

Indeed, this is probably just one aspect of what is discussed there. We may close this as a duplicate.

I had searched for existing issues but since the title of the other issue doesn't really describe the problem I couldn't find anything.

[uranusjr]

This is actually not (directly) related to #8711, because editables are weird and need special treatment. The solution (which only applies to editable) is also briefly discussed in #9147 (linked above).

[bluenote10]

Is "local development" not the same as "editable"? Since #8711 misses a concrete issue description I assumed it refers to pip install -e as well...

[pfmoore]

Is "local development" not the same as "editable"?

Not necessarily. Making changes and then reinstalling, and never using -e at all, is a perfectly valid workflow (and one I use sometimes). One of the reasons people might use it is precisely because editables are a bit weird at times 🙂

[pradyunsg]

FWIW, this is one my things-to-do-before-release list, as noted in #9147 (comment).