Poetry Virtual environment breaks shared namespaces #9684
Comments
FlowMatric
added
kind/bug
|
Poetry is doing an editable install. "The same way as pip" is not a goal, but Though nb there is more than one way to do an editable install, I forget which pip is likely to do or when that behaviour changed. |
|
Also the way you have set up the Poetry itself makes use of namespace packages, you likely want to do the same thing that it and poetry-core do |
|
I'm glad I'm not the only one facing this 😅 If it helps anyone else searching, removing... from pyproject.toml restored the correct behavior for packages with shared namespaces. You lose PEP 517 builds but the repo works again. |
|
the right fix in the example of this issue is to correct the |
|
@dimbleby do you have any examples you can point us to with the correct setup? All examples I have found online thusfar result in the same issue described above. |
|
read up
|
@dimbleby , thanks for your input so far. I am a bit at a loss now. You are right that it works perfectly fine with the configuration you proposed (see branch package_setup_2). I don't get it though because this is the setup that I started out with in a real-world project. I'll have to go back and better understand what is going on. Maybe it's OK to keep the issue open until I can confirm that there is NO issue? |
|
Thanks for the direct link @dimbleby! That wasn't the part I had an issue with but was able to figure out a stray @FlowMatric I found my difference by diffing the result of this script below with my existing env and found a lingering link in the poetry-managed venv in
Repro Script #!/bin/bash
set -euxo pipefail
TEMP_DIR=$(mktemp -d)
mkdir -p $TEMP_DIR/package_a/src/namespace/a
mkdir -p $TEMP_DIR/package_b/src/namespace/b
mkdir -p $TEMP_DIR/package_c/src/namespace/c
touch $TEMP_DIR/package_a/src/namespace/a/__init__.py
touch $TEMP_DIR/package_b/src/namespace/b/__init__.py
touch $TEMP_DIR/package_c/src/namespace/c/__init__.py
touch $TEMP_DIR/package_c/src/namespace/__init__.py # <-- this was the problem
cat >> $TEMP_DIR/pyproject.toml <<EOF
[tool.poetry]
name = "PACKAGE_NAME_HERE"
version = "0.1.0"
description = "Test"
authors = ["Patrick Hulce <[email protected]>"]
packages = [
{ include = "INCLUDE_PATH", from = "src" },
]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
[tool.poetry.dependencies]
python = "==3.11.*"
EOF
cp $TEMP_DIR/pyproject.toml $TEMP_DIR/package_a/pyproject.toml
echo "VALUE_A = 1" >> $TEMP_DIR/package_a/src/namespace/a/__init__.py
sed -i '' 's/PACKAGE_NAME_HERE/namespace-a/g' $TEMP_DIR/package_a/pyproject.toml
sed -i '' 's#INCLUDE_PATH#namespace/a#g' $TEMP_DIR/package_a/pyproject.toml
cp $TEMP_DIR/pyproject.toml $TEMP_DIR/package_b/pyproject.toml
echo "import namespace.a" >> $TEMP_DIR/package_b/src/namespace/b/__init__.py
echo "VALUE_B = 2" >> $TEMP_DIR/package_b/src/namespace/b/__init__.py
sed -i '' 's/PACKAGE_NAME_HERE/namespace-b/g' $TEMP_DIR/package_b/pyproject.toml
sed -i '' 's#INCLUDE_PATH#namespace/b#g' $TEMP_DIR/package_b/pyproject.toml
echo '"namespace.a" = { path = "../package_a", develop = true }' >> $TEMP_DIR/package_b/pyproject.toml
cp $TEMP_DIR/pyproject.toml $TEMP_DIR/package_c/pyproject.toml
touch $TEMP_DIR/package_c/src/namespace/c/__init__.py
echo "import namespace.a" >> $TEMP_DIR/package_c/src/namespace/c/run.py
echo "import namespace.b" >> $TEMP_DIR/package_c/src/namespace/c/run.py
echo "print(namespace.a.VALUE_A)" >> $TEMP_DIR/package_c/src/namespace/c/run.py
echo "print(namespace.b.VALUE_B)" >> $TEMP_DIR/package_c/src/namespace/c/run.py
sed -i '' 's/PACKAGE_NAME_HERE/namespace-c/g' $TEMP_DIR/package_c/pyproject.toml
sed -i '' 's#INCLUDE_PATH#namespace/c#g' $TEMP_DIR/package_c/pyproject.toml
echo '"namespace.a" = { path = "../package_a", develop = true }' >> $TEMP_DIR/package_c/pyproject.toml
echo '"namespace.b" = { path = "../package_b", develop = true }' >> $TEMP_DIR/package_c/pyproject.toml
set +e
cd $TEMP_DIR/package_b
poetry install
poetry run python -c 'import namespace.a; print(namespace.a.VALUE_A)'
poetry run python -c 'import namespace.b; print(namespace.b.VALUE_B)'
cd $TEMP_DIR/package_c
poetry install
poetry run python -c 'import namespace.a; print(namespace.a.VALUE_A)'
poetry run python -c 'import namespace.b; print(namespace.b.VALUE_B)'
poetry run python -m namespace.c.run
poetry run python src/namespace/c/run.pyWith Accidental Extra init.py Without Accidental Extra init.py |
Description
I found that
poetry installcreates a virtual environment (or handles dependencies) not the same way as an environment managed by pip. IMO, poetry should be able to handle shared high level namespaces, the same way as pip does:When a poetry-managed project uses a namespace that is also used by a dependency, the project code in sub-packages is no longer available in the project
.venv.In contrast, when one creates a virtual environment with venv, differently named sub-packages are "on the path"
Here's the example project I created for this issue: https://github.com/FlowMatric/namespace_test
Workarounds
None
Poetry Installation Method
pipx
Operating System
Ubuntu 22.04
Poetry Version
1.8.3
Poetry Configuration
Python Sysconfig
Example pyproject.toml
Please see package configs in [this example project](https://github.com/FlowMatric/namespace_test)Poetry Runtime Logs
The text was updated successfully, but these errors were encountered: