You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #12063 support for non-standard (ie: broken) dependency specifiers was first deprecated, and then in pip 24.1.0, removed entirely.
As such pip now ignores such broken packages when resolving dependencies during package installation - meaning they can no longer be installed. This is fine, and currently working as expected.
However, what it not working as expected, is that if there are any already-installed dependencies with those dependency specifiers then pip install fails with an internal error and does not explain what happened or how to resolve it.
Expected behavior
Using the steps to reproduce below, pip should either:
Succeed as a no-op, since the requested dependency is satisfied (ie: there is nothing to install/uninstall), since "celery" is satisfied by any installed version. (And in fact the log output below even says "Requirement already satisfied".)
Or, successfully uninstall the broken package and install a non-broken version (which for Celery, is 5.3.1+).
Or error, but do so with a helpful error message (explaining broken packages are installed and to either manually remove those packages or downgrade pip), rather than show an unhandled internal error/traceback.
pip version
24.2
Python version
3.12.5 (homebrew)
OS
macOS
How to Reproduce
mkdir testcase && cd $_
python -m venv .venv && source .venv/bin/activate
pip install pip==24.0
pip install celery==4.4.7
pip install pip==24.2
pip install celery
Output
(.venv) testcase $ pip install celery
Requirement already satisfied: celery in ./.venv/lib/python3.12/site-packages (4.4.7)
ERROR: Exception:
Traceback (most recent call last):
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/requirements.py", line 36, in __init__
parsed = _parse_requirement(requirement_string)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_parser.py", line 62, in parse_requirement
return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_parser.py", line 80, in _parse_requirement
url, specifier, marker = _parse_requirement_details(tokenizer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_parser.py", line 118, in _parse_requirement_details
specifier = _parse_specifier(tokenizer)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_parser.py", line 208, in _parse_specifier
with tokenizer.enclosing_tokens(
File "/opt/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 144, in __exit__
next(self.gen)
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_tokenizer.py", line 189, in enclosing_tokens
self.raise_syntax_error(
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/_tokenizer.py", line 167, in raise_syntax_error
raise ParserSyntaxError(
pip._vendor.packaging._tokenizer.ParserSyntaxError: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
pytz (>dev)
~^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
status = _inner_run()
^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
return self.run(options, args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 379, in run
requirement_set = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
result = self._result = resolver.resolve(
^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
state = resolution.resolve(requirements, max_rounds=max_rounds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
failure_causes = self._attempt_to_pin_criterion(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
criteria = self._get_updated_criteria(candidate)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 229, in _get_updated_criteria
for requirement in self._p.get_dependencies(candidate=candidate):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 247, in get_dependencies
return [r for r in candidate.iter_dependencies(with_requires) if r is not None]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 401, in iter_dependencies
for r in self.dist.iter_dependencies():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py", line 215, in iter_dependencies
req = get_requirement(req_string.strip())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_internal/utils/packaging.py", line 45, in get_requirement
return Requirement(req_string)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/emorley/src/testcase/.venv/lib/python3.12/site-packages/pip/_vendor/packaging/requirements.py", line 38, in __init__
raise InvalidRequirement(str(e)) from e
pip._vendor.packaging.requirements.InvalidRequirement: Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
pytz (>dev)
~^
This blocks Heroku upgrading from pip 24.0 to something newer (eg heroku/heroku-buildpack-python#1619), since we will no doubt have a long tail of users with existing cached site-packages with packages using non-standard dependency specifiers installed. (Whilst users should be upgrading to newer versions of packages with those specifiers fixed, if there isn't a clear error message, they won't know what the issue is or how to resolve it.)
Note: This issue seems similar to #12945, though that issue seems more about initial installation, rather than "existing installation", plus doesn't have explicit steps to reproduce - so it seemed best to file a separate issue.
edmorley
changed the title
Unhandled error from already satisified dependency with non-standard dependency specifier
Unhandled error from already installed dependency with non-standard dependency specifier
Sep 2, 2024
edmorley
changed the title
Unhandled error from already installed dependency with non-standard dependency specifier
Unhandled error from already-installed dependency with non-standard dependency specifier
Sep 2, 2024
I've created #12953 that gives a much clearer error message:
$ python -m pip install --dry-run celery
Requirement already satisfied: celery in ./.venv/lib/python3.12/site-packages (4.4.7)
error: invalid-installed-package
× Cannot process installed package celery 4.4.7 in '/home/damian/opensource/contributions/pip/.venv/lib/python3.12/site-packages' because it has an invalid requirement:
│ Expected matching RIGHT_PARENTHESIS for LEFT_PARENTHESIS, after version specifier
│ pytz (>dev)
│ ~^
╰─> Starting with pip 24.1, packages with invalid requirements can not be processed.
hint: To proceed this package must be uninstalled using 'pip<24.1', some other Python package tool, or manually deleted.
Edit: Updated message to reflect latest version of PR
Description
In #12063 support for non-standard (ie: broken) dependency specifiers was first deprecated, and then in pip 24.1.0, removed entirely.
As such pip now ignores such broken packages when resolving dependencies during package installation - meaning they can no longer be installed. This is fine, and currently working as expected.
However, what it not working as expected, is that if there are any already-installed dependencies with those dependency specifiers then pip install fails with an internal error and does not explain what happened or how to resolve it.
Expected behavior
Using the steps to reproduce below, pip should either:
"celery"
is satisfied by any installed version. (And in fact the log output below even says "Requirement already satisfied".)pip version
24.2
Python version
3.12.5 (homebrew)
OS
macOS
How to Reproduce
mkdir testcase && cd $_
python -m venv .venv && source .venv/bin/activate
pip install pip==24.0
pip install celery==4.4.7
pip install pip==24.2
pip install celery
Output
Code of Conduct
The text was updated successfully, but these errors were encountered: