poetry install fails when tests are in a separate subdir from the main package

[abadger]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• Fedora 31:
• Poetry version: 1.0.5:
• Tarball with pyproject.toml and directory layout ** https://toshio.fedorapeople.org/trees.tar.gz **:

Issue

When a pyproject.toml has multiple include entries in the packages list and one of those entries is listed as format='sdist', poetry will mistakenly try to install that package into the virtualenv when doing poetry install. In conjunction with: #1811 this will lead to a traceback.

Reproducer:

$ wget https://toshio.fedorapeople.org/trees.tar.gz                                  (01:08:06)
$ tar -xzf trees.tar.gz                                                             (01:08:12)
$ cd trees                                                                           (01:08:16)
$ poetry install                                                               (01:08:20)
Installing dependencies from lock file

No dependencies to install or update

  - Installing trees (0.1.0)

[EnvCommandError]
Command ['/home/badger/.cache/pypoetry/virtualenvs/trees-qCG1GaiB-py3.7/bin/pip', 'install', '-e', '/var/tmp/trees'] errored with the following return code 1, and output: 
Obtaining file:///var/tmp/trees
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: running egg_info
    writing src/trees.egg-info/PKG-INFO
    writing dependency_links to src/trees.egg-info/dependency_links.txt
    writing top-level names to src/trees.egg-info/top_level.txt
    error: package directory 'src/test' does not exist
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /var/tmp/trees/
WARNING: You are using pip version 19.1.1, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.


Traceback (most recent call last):
  File "/home/badger/.local/lib/python3.7/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/home/badger/.local/lib/python3.7/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/home/badger/.local/lib/python3.7/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/home/badger/.local/lib/python3.7/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/console/commands/install.py", line 88, in handle
    builder.build()
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/masonry/builders/editable.py", line 17, in build
    return self._setup_build()
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/masonry/builders/editable.py", line 41, in _setup_build
    self._env.run_pip('install', '-e', str(self._path))
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/utils/env.py", line 861, in run_pip
    return self._run(cmd, **kwargs)
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/utils/env.py", line 1089, in _run
    return super(VirtualEnv, self)._run(cmd, **kwargs)
  File "/home/badger/.local/lib/python3.7/site-packages/poetry/utils/env.py", line 893, in _run
    raise EnvCommandError(e, input=input_)

Relevant portion of pyproject.toml from the reproducer tarball:

packages = [
        {include="apple", from="src"},
        {include="test", format="sdist"},
]

The problem is that poetry is generating a setup.py to be able to pip install -e. The setup.py entry for packages is wrong:

package_dir = \
{'': 'src'}

packages = \
['apple', 'test']

In this case, because {include="test"[..]} has format="sdist", test should not appear in the packages list.

[abadger]

I saw that there had been some changes in the develop branch to installing the root package. (Is that the right jargon for the package that I'm using poetry to build?) I installed the develop branch and gave this another try. The good news is that on the develop branch poetry install will complete.

The bad news is that it doesn't install correctly. It seems that it still only knows about one of the from entries in the package list. When it installs, it creates a .pth file in the virtualenv. But it can contain the wrong path:

$ cat /home/badger/.cache/pypoetry/virtualenvs/trees-qCG1GaiB-py3.8/lib/python3.8/site-packages/trees.pth
/var/tmp/trees

For this test case, the .pth file should contain /var/tmp/trees/src

Poetry commit hash on develop: abee30f

[neersighted]

I saw that there had been some changes in the develop branch to installing the root package. (Is that the right jargon for the package that I'm using poetry to build?) I installed the develop branch and gave this another try. The good news is that on the develop branch poetry install will complete.

The bad news is that it doesn't install correctly. It seems that it still only knows about one of the from entries in the package list. When it installs, it creates a .pth file in the virtualenv. But it can contain the wrong path:

$ cat /home/badger/.cache/pypoetry/virtualenvs/trees-qCG1GaiB-py3.8/lib/python3.8/site-packages/trees.pth
/var/tmp/trees

For this test case, the .pth file should contain /var/tmp/trees/src

Poetry commit hash on develop: abee30f

Hey @abadger -- would you be willing to try with the latest poetry-core? I do believe that the from= issue should be fixed and that hopefully will solve the second half of this issue.

[abadger]

I'll try to give it a look this weekend

…

On Tue, Nov 9, 2021, 10:05 PM Bjorn Neergaard ***@***.***> wrote: I saw that there had been some changes in the develop branch to installing the root package. (Is that the right jargon for the package that I'm using poetry to build?) I installed the develop branch and gave this another try. The good news is that on the develop branch poetry install will complete. The bad news is that it doesn't install correctly. It seems that it still only knows about one of the from entries in the package list. When it installs, it creates a .pth file in the virtualenv. But it can contain the wrong path: $ cat /home/badger/.cache/pypoetry/virtualenvs/trees-qCG1GaiB-py3.8/lib/python3.8/site-packages/trees.pth /var/tmp/trees For this test case, the .pth file should contain /var/tmp/trees/src Poetry commit hash on develop: abee30f <abee30f> Hey @abadger <https://github.com/abadger> -- would you be willing to try with the latest poetry-core? I do believe that the from= issue should be fixed and that hopefully will solve the second half of this issue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2450 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABTCWRLGUXRBR2J3Z33H4TULIDTHANCNFSM4NG6IGIQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[abadger]

I have tested and it seems to work now! Thanks!

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.