poetry run pip install . doesn't work with directory dependencies

[AndrewGuenther]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

No verbose output because -v is currently broken: #3751

• OS version and name: Ubuntu 20.04
• Poetry version: 1.1.5
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/AndrewGuenther/4fc4bb77bad367655bcdd148587fac8e

Issue

When I try to run poetry run pip install . for my poetry package, it breaks if I have a directory dependency with the following error:

Processing /home/andrew/worksapce/projects/my-project
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/bin/python /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0h3jkq4v
         cwd: /tmp/pip-req-build-5e4zvq59
    Complete output (16 lines):
    Traceback (most recent call last):
      File "/home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-ta34h747/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 43, in prepare_metadata_for_build_wheel
        poetry = Factory().create_poetry(Path(".").resolve(), with_dev=False)
      File "/tmp/pip-build-env-ta34h747/overlay/lib/python3.8/site-packages/poetry/core/factory.py", line 93, in create_poetry
        self.create_dependency(name, constraint, root_dir=package.root_dir)
      File "/tmp/pip-build-env-ta34h747/overlay/lib/python3.8/site-packages/poetry/core/factory.py", line 244, in create_dependency
        dependency = DirectoryDependency(
      File "/tmp/pip-build-env-ta34h747/overlay/lib/python3.8/site-packages/poetry/core/packages/directory_dependency.py", line 41, in __init__
        raise ValueError("Directory {} does not exist".format(self._path))
    ValueError: Directory ../../lib/my-project-lib does not exist
    ----------------------------------------
WARNING: Discarding file:///home/andrew/worksapce/projects/my-project. Command errored out with exit status 1: /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/bin/python /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0h3jkq4v Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/bin/python /home/andrew/.cache/pypoetry/virtualenvs/my-project-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0h3jkq4v Check the logs for full command output.

../../lib/my-project-lib does exist, and my package can build just fine when running poetry build. Executing poetry run pip install ../../lib/my-project-lib or poetry add ../../lib/my-project-lib also work. Seems like there's some local dependency resolution issues happening specifically when poetry is invoked by pip.

[AndrewGuenther]

I tried changing my relative path reference to an absolute one and I ran into this error:

Traceback (most recent call last):
  File "/home/andrew/.cache/pypoetry/virtualenvs/data-pipeline-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3021, in _dep_map
    return self.__dep_map
  File "/home/andrew/.cache/pypoetry/virtualenvs/data-pipeline-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 2815, in __getattr__
    raise AttributeError(attr)
AttributeError: _DistInfoDistribution__dep_map

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/andrew/.cache/pypoetry/virtualenvs/data-pipeline-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/pkg_resources/__init__.py", line 3102, in __init__
    super(Requirement, self).__init__(requirement_string)
  File "/home/andrew/.cache/pypoetry/virtualenvs/data-pipeline-JuauOX3v-py3.8/lib/python3.8/site-packages/pip/_vendor/packaging/requirements.py", line 130, in __init__
    raise InvalidRequirement("Invalid URL: {0}".format(req.url))
pip._vendor.packaging.requirements.InvalidRequirement: Invalid URL: /home/andrew/worksapce/lib/my-project-lib

So this got me past the issue internal to poetry which would appear to be the result of some path mishandling. This issue is now internal to pip, but some logging reveals that poetry is passing pip a requirement string of the form

my-project-lib @ /home/andrew/worksapce/lib/my-project-lib

Which isn't valid. Same root cause as the issue fixed by this PR #3121 , but this issue is actually in core.

[AndrewGuenther]

So, there's two issues here. The issue mentioned in my comment above about absolute path dependencues is fixed by python-poetry/poetry-core#141. The other part of this, relative paths, is actually an issue with how pip handles PEP 517 builds. I've filed and issue with them pypa/pip#9716

TL;DR on that: pip moves your package into a temp directory before invoking the build backend and doesn't tell you where you came from. As a result, you can't resolve a relative path because you have no context on what it is actually relative to.

[finswimmer]

Hello @AndrewGuenther,

as you figured out, the issue with the absolute path is already solved. This issue with relative path exists, because there is no definition what "relative" is exactly. Relative to the path where pip install (or any other comment) was invoked? Relative to the current working directory? Relative to the source of the packe? ...

Path dependencies are fine for testing or if you have checked out the whole project and install it in editable mode. If you want to provide a "real" package (sdist, wheel) avoid path dependencies.

I think it's fine to close this issue as one part is already solve and the other cannot be solved?

fin swimmer

[AndrewGuenther]

I don't think this is quite resolved. My PR resolves part of it, but still won't work for relative paths. Pip rejected my issue saying that is non-standard. Should poetry have clear messaging when invoked via PEP 517 builds that it will fail due to a relative path? I'd imaging an error both on that path and poetry export letting folks know that it will not work.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.