Use the "venv" scheme if available to obtain prefixed lib paths #11598
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dnicolodi
force-pushed
the
homebrew-scheme
branch
4 times, most recently
from
90ae2c9
to
418771a
Compare
dnicolodi
added a commit
to dnicolodi/meson-python
that referenced
this pull request
Nov 16, 2022
dnicolodi
added a commit
to dnicolodi/meson-python
that referenced
this pull request
Nov 16, 2022
uranusjr
reviewed
Nov 16, 2022
Comment on lines
216
to
222
| def get_prefixed_libs(prefix: str) -> typing.Tuple[str, str]: | ||
| paths = sysconfig.get_paths(vars={"base": prefix, "platbase": prefix}) | ||
| vars = {"base": prefix, "platbase": prefix} | ||
| if "venv" in sysconfig.get_scheme_names(): | ||
| paths = sysconfig.get_paths(vars=vars, scheme="venv") | ||
| else: | ||
| paths = sysconfig.get_paths(vars=vars) | ||
| return (paths["purelib"], paths["platlib"]) |
There was a problem hiding this comment.
Since this function is only used for creating isolated environments, let’s rename it to better describe what it does. This avoids needing to think about why this always sets venv even when pip is not running in a virtual environment.
There was a problem hiding this comment.
I thought prefixed in the name carried this message. Do you have a better name in mind?
There was a problem hiding this comment.
Maybe just get_isolated_environment_lib_paths?
There was a problem hiding this comment.
Sounds descriptive enough. Should I change the argument name to environment then?
There was a problem hiding this comment.
prefix is probably fine (it is the environment’s path prefix after all)
There was a problem hiding this comment.
I just noticed that the function name is meant to reflect the name of a similar function in distutils. It is also called from a function with the same name in another module, see
pip/src/pip/_internal/locations/__init__.py
Lines 485 to 489 in 9aa422d
1 task
FFY00
pushed a commit
to mesonbuild/meson-python
that referenced
this pull request
Nov 16, 2022
get_prefixed_libs() computes the Python path for libraries in a pip
isolation environment. Python 3.11 introduced the "venv" path scheme
to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the
default paths scheme when Python is invoked outside a virtual
environment is "osx_framework_library" and does not relative to the
"{base}" or "{platbase}" variables.
Fixes pypa#11539.
dnicolodi
force-pushed
the
homebrew-scheme
branch
from
985f850
to
0555eea
Compare
Since this function is only used for creating isolated environments, rename it to better describe what it does. This avoids needing to think about why the implementation uses the "venv" paths scheme even when pip is not running in a virtual environment.
dnicolodi
force-pushed
the
homebrew-scheme
branch
from
0555eea
to
f8beb61
Compare
uranusjr
approved these changes
Nov 17, 2022
|
This also fixes #11053 |
pradyunsg
approved these changes
Nov 18, 2022
|
I've triggered a CI re-run for some flakiness. |
|
All green now. Can this be merged or there is anything left to do? |
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 27, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 29, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
dnicolodi
added a commit
to dnicolodi/pip
that referenced
this pull request
Nov 29, 2022
The scripts path was looked up passing explicitly the scheme to be used using "nt" on Windows and "posix_prefix" everywhere else. However, when the isolated build environment is created, packages are installed using the default scheme for the platform. On most platforms this works because normally "nt" and "posix_prefix" are the default schemes. However, Debian customizes sysconfig to use a "posix_local" scheme by default and under this scheme the scripts path does not match the one of the "posix_prefix" scheme. This results in scripts installed as part of the build dependencies not to be found during the build, as reported here mesonbuild/meson-python#109 and here https://bugs.debian.org/1019293. The problem can be solved omitting to specify a scheme when looking up the scripts path. To future proof the path lookup, use the "venv" scheme if available as done in pypa#11598. For uniformity use similar functions as used to lookup the library paths.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
get_prefixed_libs() computes the Python path for libraries in a pip isolation environment. Python 3.11 introduced the "venv" path scheme to be used in these cases. Use it if available.
This solves a bug on Homebrew's Python 3.10 and later where the default paths scheme when Python is invoked outside a virtual environment is "osx_framework_library" and does not relative to the "{base}" or "{platbase}" variables.
Fixes #11539.