Broken Relative Paths when package in subtree of installing package

[jamtat]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS/Linux (windows untested)
• Poetry version: 1.1.17
• Link of a Gist with the contents of your pyproject.toml file: https://github.com/jamtat/poetry-broken-relative-paths

Issue

(copy+pasted from above repo linked)

Broken Relative Paths when package in subtree of installing package

Relative path imports fail when there is a dependency tree such that:

• application -> otherlib -> baselib
• otherlib and baselib share a filepath with pyproject.toml such that if pyproject.toml is at <path>/pyproject.toml, other libs are at <path>/<any descendant path>/{otherlib,baselib}

Steps to reproduce

• Have poetry 1.1.17
• Latest pip
• Run:

pushd common/baselib; poetry install; popd
pushd common/otherlib; poetry install; popd
poetry install

Expected result

• In the case that having a package be in the subtree of another package is supported the relative path should be properly adjusted to be relative to the pyproject.toml
• In this case that this behaviour is unsupported a more descriptive error should be raised, possibly referencing this issue and saying something like "Packages may not exist as a subtree of a dependant package"

Result

  Stack trace:

  11  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  10  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

   9  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:163 in _do_handle
       161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
       162│             event = PreHandleEvent(args, io, self)
     → 163│             self._dispatcher.dispatch(PRE_HANDLE, event)
       164│ 
       165│             if event.is_handled():

   8  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
        20│ 
        21│         if listeners:
     →  22│             self._do_dispatch(listeners, event_name, event)
        23│ 
        24│         return event

   7  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
        87│                 break
        88│ 
     →  89│             listener(event, event_name, self)
        90│ 
        91│     def _sort_listeners(self, event_name):  # type: (str) -> None

   6  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/console/config/application_config.py:116 in set_env
       114│ 
       115│         io = event.io
     → 116│         poetry = command.poetry
       117│ 
       118│         env_manager = EnvManager(poetry)

   5  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/console/commands/command.py:10 in poetry
        8│     @property
        9│     def poetry(self):
     → 10│         return self.application.poetry
       11│ 
       12│     def reset_poetry(self):  # type: () -> None

   4  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/console/application.py:69 in poetry
        67│             return self._poetry
        68│ 
     →  69│         self._poetry = Factory().create_poetry(Path.cwd())
        70│ 
        71│         return self._poetry

   3  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/factory.py:33 in create_poetry
        31│             io = NullIO()
        32│ 
     →  33│         base_poetry = super(Factory, self).create_poetry(cwd)
        34│ 
        35│         locker = Locker(

   2  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/core/factory.py:93 in create_poetry
        91│ 
        92│                 package.add_dependency(
     →  93│                     self.create_dependency(name, constraint, root_dir=package.root_dir)
        94│                 )
        95│ 

   1  ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/core/factory.py:251 in create_dependency
       249│                         base=root_dir,
       250│                         develop=constraint.get("develop", False),
     → 251│                         extras=constraint.get("extras", []),
       252│                     )
       253│             elif "url" in constraint:

  ValueError

  Directory ../common/otherlib does not exist

  at ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/core/packages/directory_dependency.py:41 in __init__
       37│         self._develop = develop
       38│         self._supports_poetry = False
       39│ 
       40│         if not self._full_path.exists():
    →  41│             raise ValueError("Directory {} does not exist".format(self._path))
       42│ 
       43│         if self._full_path.is_file():
       44│             raise ValueError("{} is a file, expected a directory".format(self._path))
       45│ 

Directory Structure:

|- pyproject.toml
|- application/
   |- __init__.py
|- common
   |- baselib/
      |- pyproject.toml
      |- baselib/
         |- __init__.py
   |- otherlib/
      |- pyproject.toml
      |- otherlib/
         |- __init__.py

[gc-ss]

To clarify, while both baselib, otherlib are siblings, they are both nested under a root project (the first pyproject.toml).

Context: https://discord.com/channels/487711540787675139/487711540787675143/872859997896319037

[mfulgo]

I believe this is a duplicate of #4245, which was just fixed by #4246.

[jamtat]

Trying it on the test repo with 1.20a2 (edit: also tried with latest master) hasn't actually fixed the issue, nor has it fixed it in the real repo inspiring the issue. I did however try downgrading to 1.1.6 which did create a working lockfile that could be installed without issue.

(upgrading also introduced an unrelated problem where some python semver strings in some package's setup.py now fail verification, I'll make a new issue for that)

[hckr]

@jamtat I believe you have a typo in your outermost pyproject.toml: https://github.com/jamtat/poetry-broken-relative-paths/blob/master/pyproject.toml#L9

I think it should be ./common/otherlib (single dot at the beginning, I'm not sure about the trailing slash) and not ../common/otherlib/.

[jamtat]

@hckr you're totally right, my mistake there.

Correcting the pyproject.toml does cause poetry to successfully install the root project on 1.20a2, however it fails on 1.1.17.

This bug does seem to be fixed. Thanks for all your help.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.