system-git-client fails to install from Pull Request

[pmav99]

Poetry
Version: 1.7.1
Python:  3.11.5

Virtualenv
Python:         3.11.5
Implementation: CPython
Path:           /tmp/asdf/.venv
Executable:     /tmp/asdf/.venv/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.11.5
Path:       /usr
Executable: /usr/bin/python3.11

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

There seem to be some regression. poetry seems to fail installing a Pull request, even though in the past it was able to do so. For example #5130 and #3240

To replicate try:

poetry init -n
poetry add 'git+https://github.com/dengwirda/inpoly-python.git@refs/pull/17/merge' -vvv

The output of poetry add is:

$ poetry add 'git+https://github.com/dengwirda/inpoly-python.git@refs/pull/17/merge' -vvv
Trying to detect current active python executable as specified in the config.
Found: /tmp/asdf/.venv/bin/python
Using virtualenv: /tmp/asdf/.venv
Cloning 'https://github.com/dengwirda/inpoly-python.git' using system git client

  Stack trace:

  4  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:243 in _clone_legacy
      241│ 
      242│         try:
    → 243│             SystemGit.checkout(revision, target)
      244│         except CalledProcessError:
      245│             raise PoetryConsoleError(f"Failed to checkout {url} at '{revision}'")

  3  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/system.py:26 in checkout
       24│     def checkout(cls, rev: str, target: Path | None = None) -> str:
       25│         cls._check_parameter(rev)
    →  26│         return cls.run("checkout", rev, folder=target)
       27│ 
       28│     @staticmethod

  2  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/system.py:43 in run
       41│         env = os.environ.copy()
       42│         env["GIT_TERMINAL_PROMPT"] = "0"
    →  43│         return subprocess.check_output(
       44│             git_command + list(args),
       45│             stderr=subprocess.STDOUT,

  1  /usr/lib/python3.11/subprocess.py:466 in check_output
       464│         kwargs['input'] = empty
       465│ 
    →  466│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
       467│                **kwargs).stdout
       468│ 

  CalledProcessError

  Command '['git', '--git-dir', '/tmp/asdf/.venv/src/inpoly-python/.git', '--work-tree', '/tmp/asdf/.venv/src/inpoly-python', 'checkout', 'refs/pull/17/merge']' returned non-zero exit status 1.

  at /usr/lib/python3.11/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│ 
       575│ 

The following error occurred when trying to handle this error:


  Stack trace:

  17  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  16  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  15  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

  14  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

  13  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  12  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  11  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  10  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/commands/add.py:160 in handle
       158│             return 0
       159│ 
     → 160│         requirements = self._determine_requirements(
       161│             packages,
       162│             allow_prereleases=self.option("allow-prereleases"),

   9  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/commands/init.py:375 in _determine_requirements
       373│ 
       374│         result = []
     → 375│         for requirement in self._parse_requirements(requires):
       376│             if "git" in requirement or "url" in requirement or "path" in requirement:
       377│                 result.append(requirement)

   8  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/commands/init.py:441 in _parse_requirements
       439│             cwd=cwd,
       440│         )
     → 441│         return [parser.parse(requirement) for requirement in requirements]
       442│ 
       443│     def _format_requirements(self, requirements: list[dict[str, str]]) -> Requirements:

   7  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/console/commands/init.py:441 in <listcomp>
       439│             cwd=cwd,
       440│         )
     → 441│         return [parser.parse(requirement) for requirement in requirements]
       442│ 
       443│     def _format_requirements(self, requirements: list[dict[str, str]]) -> Requirements:

   6  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/utils/dependency_specification.py:89 in parse
        87│ 
        88│         specification = (
     →  89│             self._parse_url(requirement)
        90│             or self._parse_path(requirement)
        91│             or self._parse_simple(requirement)

   5  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/utils/dependency_specification.py:149 in _parse_url
       147│ 
       148│         if url_parsed.scheme in GIT_URL_SCHEMES:
     → 149│             return self._parse_git_url(requirement)
       150│ 
       151│         if url_parsed.scheme in ["http", "https"]:

   4  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/utils/dependency_specification.py:133 in _parse_git_url
       131│ 
       132│         source_root = self._env.path.joinpath("src") if self._env else None
     → 133│         package = self._direct_origin.get_package_from_vcs(
       134│             "git",
       135│             url=url.url,

   3  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/packages/direct_origin.py:106 in get_package_from_vcs
       104│             raise ValueError(f"Unsupported VCS dependency {vcs}")
       105│ 
     → 106│         return _get_package_from_git(
       107│             url=url,
       108│             branch=branch,

   2  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/packages/direct_origin.py:32 in _get_package_from_git
        30│     source_root: Path | None = None,
        31│ ) -> Package:
     →  32│     source = Git.clone(
        33│         url=url,
        34│         source_root=source_root,

   1  ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:473 in clone
       471│ 
       472│         # fallback to legacy git client
     → 473│         return cls._clone_legacy(url=url, refspec=refspec, target=target)
       474│ 
       475│ 

  PoetryConsoleError

  Failed to checkout https://github.com/dengwirda/inpoly-python.git at 'refs/pull/17/merge'

  at ~/.local/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/vcs/git/backend.py:245 in _clone_legacy
      241│ 
      242│         try:
      243│             SystemGit.checkout(revision, target)
      244│         except CalledProcessError:
    → 245│             raise PoetryConsoleError(f"Failed to checkout {url} at '{revision}'")
      246│ 
      247│         repo = Repo(str(target))
      248│         return repo
      249│ 

Judging from:

  Command '['git', '--git-dir', '/tmp/asdf/.venv/src/inpoly-python/.git', '--work-tree', '/tmp/asdf/.venv/src/inpoly-python', 'checkout', 'refs/pull/17/merge']' returned non-zero exit status 1.

It seems that the refs have not been fetched from github.

[dimbleby]

works fine if you don't disable the dulwich git client

I guess nothing has changed, probably #3240 was closed because this does work with dulwich rather than because it ever worked with the system git client

[pmav99]

Indeed I was using the system git client and I do confirm that it works if I switch to dulwitch. Nevertheless the colleague who asked me for help is using dulwitch and he couldn't install it. Not sure what happened.

Anyhow, if you just directly edit pyproject.toml it works even with the old client: inpoly = {git = "https://github.com/dengwirda/inpoly-python.git", rev = "refs/pull/17/merge"}

I will update the issue description to reflect that this issue affects the system git client.

[pmav99]

Something like this seems to fix it. It's a bit hackish though. It would probably be more consistent if dedicated method was added to the SystemGit class.

$ git diff
diff --git a/src/poetry/vcs/git/backend.py b/src/poetry/vcs/git/backend.py
index 4b7d9872..d4238903 100644
--- a/src/poetry/vcs/git/backend.py
+++ b/src/poetry/vcs/git/backend.py
@@ -234,12 +232,14 @@ class Git:
             revision.replace("refs/head/", "")
             revision.replace("refs/tags/", "")
 
+        if "refs/pull" in revision:
+            SystemGit.run("fetch", "origin", revision, folder=target)
+            revision = "FETCH_HEAD"
+
         try:
             SystemGit.checkout(revision, target)
         except CalledProcessError: