Flag to allow uv pip install to install into specific environments (that do not necessarily contain uv)

[hauntsaninja]

It looks like uv already supports this, I just need to lie about VIRTUAL_ENV.

Looking for something like the equivalent of pip's --prefix or --python.

[RonNabuurs]

I did also encounter this when trying to use uv as a drop in replacement for my pip projects.

[ofek]

Definitely agree, this is fundamental to allow a single UV installation to be used as a real standalone binary!

[zanieb]

Yes we should support this. Is there a consensus on the name of the flag in other tools?

[ericbn]

pip has the --require-virtualenv option which makes it behave like uv:

$ pip install --require-virtualenv attrs
ERROR: Could not find an activated virtualenv (required).

but --no-require-virtualenv seems too long of a name for an option in uv. And sure the default uv behavior is much saner default!

[hauntsaninja]

pip uses --prefix and --python (I think the difference relates to which Python pip should be run with, which doesn't matter to uv).
pypa/installer uses --prefix.

I'm not too familiar with pdm, but the --venv flag several of its command support is a name not a path (which is a good choice for high level workflow tool).

Meta since I got confused:

• There's maybe some overlap between this issue and Cannot uv pip install without virtualenv #1374 and Add flag to allow global installs e.g. for CI #1526 (which are duplicates of each other). In both cases the workaround is to set VIRTUAL_ENV explicitly.
• I view this issue as low priority and just providing a flag that does a very mechanical equivalent of VIRTUAL_ENV=path uv pip install ... via uv pip install --prefix=path ...
• The other issues is a flag that a) does something sort of like VIRTUAL_ENV=$(/path/to/base/python -c "import sys; print(sys.prefix)") uv pip install ... via uv pip install --system ..., b) promises that setting that flag will make things work for a base Python, since it's not clear how intentional it is that setting VIRTUAL_ENV currently lets you point to a base Python. Those issues also maybe need more thinking about how to identify the right base Python, e.g. if you do python -m uv ... should it look at sys.base_prefix?

[ofek]

pip uses --prefix and --python (I think the difference relates to which Python pip should be run with, which doesn't matter to uv).

A few things:

1. The --python flag is desirable here because the tool that creates an environment is not necessarily the same program that uses an environment (perhaps in IDE). In this case, there is no knowledge of the base Python that was used and therefore no way to get information about its version, architecture, etc. It's not true to say that pip uses the flag to decide where to run (it's Python so it's already running) but rather it's entirely about information gathering.

   There is no way to avoid invoking the interpreter to get that data until PEP 739 is accepted (and implemented by various tools) and then another of the same kind for virtual environments is accepted (and implemented by various tools).

2. The VIRTUAL_ENV workaround indeed does not work well for base installations because the directory structure is not guaranteed and is often different based on the platform. For example, on Windows installations usually have a python.exe at the root whereas a virtual environment will have that under the Scripts directory. That command doesn't work in the official Python Docker image for instance:

   ❯ docker run --rm python:3.12 python -c "import sys; print(sys.prefix)"
   /usr/local
   

[hauntsaninja]

@ofek no, pip will literally run itself in a subprocess using a different interpreter with --python: https://github.com/pypa/pip/blob/ebe491a82a13e6610697b22be00db363fb5ff5e3/src/pip/_internal/cli/main_parser.py#L82

[ofek]

Thanks for linking the code, I didn't realize that's how they chose to implement it! My point still stands in that pip is already running and therefore the previous assertion that it needs to know where to run is not true (except for that implementation detail) but its purpose is rather to get details about the interpreter. Other tools like virtualenv, Hatch and Poetry use a script that returns the environment of a given interpreter:

• virtualenv: https://github.com/pypa/virtualenv/blob/20.25.0/src/virtualenv/discovery/py_info.py
• Hatch: https://github.com/pypa/hatch/blob/hatch-v1.9.3/src/hatch/utils/env.py#L45-L79
• Poetry: https://github.com/python-poetry/poetry/blob/1.7.1/src/poetry/utils/env/script_strings.py#L30-L98

[hauntsaninja]

Sorry, I'm still a little bit confused about what your ask is. Note that uv does invoke the interpreter it is trying to install into in all cases:

• uv/crates/uv-interpreter/src/get_interpreter_info.py

  Line 1 in 2586f65

  import json

• uv/crates/uv-interpreter/src/virtual_env.rs

  Line 31 in 2586f65

  let interpreter = Interpreter::query(&executable, &platform.0, cache)?;

This issue really should just be adding a flag that does what the env var already does

I don't know if uv handles installing into base python on Windows correctly with the VIRTUAL_ENV trick (and that's more a question for #1374). But in the official docker image case you talk about uv does the right thing with the trick, I'm not sure why you say it doesn't work: docker run --rm python:3.12 bash -c 'pip install uv mypy; VIRTUAL_ENV=$(python -c "import sys; print(sys.prefix)") uv pip freeze'

[ofek]

• My point is that if we are provided an option for an explicit path then it would be far more rational for that option to be a Python binary which is ultimately invoked rather than a directory that must have a specific structure in order to find the aforementioned binary. The example I gave was a real one in which IDEs act upon paths to interpreters rather than directories per se.

  I didn't know UV already polls the interpreter; that's good! So it looks like just an oversight to be able to control the specific Python interpreter.

• What I was trying to show earlier was that the VIRTUAL_ENV environment variable is working by happenstance because the default sys.prefix directory /usr/local happens to have a bin directory which matches the expected Linux virtual environment structure. As you mentioned this doesn't work on Windows and I'm not certain that works for all Linux installations either.

[hauntsaninja]

Okay, I think I see! To summarise:

• Currently, you can do VIRTUAL_ENV=/path/to/prefix uv pip ...
• It would be nice if you could instead do uv pip --prefix /path/to/prefix ...
  • This would make uv install into a specified virtual environment just a tiny bit nicer, which was my use case when opening this issue.
  • Honestly, I don't even mind the env var that much, I think I just ran uv pip install --help and didn't see the pip option that I knew I could use to do this. My intention when opening issue was basically just a discoverability nit :-)
• Alternatively, the flag could be uv pip --python-executable /path/to/python ...
• --python-executable might have an advantage if you wanted this flag to be able to install into base environments, because while uv knows directory structure for venvs, it may not for base environments (see also Cannot uv pip install without virtualenv #1374 / Add flag to allow global installs e.g. for CI #1526 ). In particular, for base environment on Windows and for Linux distro provided Python (like when python is /usr/bin/python3 and dist-packages nonsense is happening or whatever)

So I guess if I were uv, I'd figure out what/how I want to do 1374 / 1526 and maybe that determines what should happen here. Most of the demand for those issues is probably stuff like "make official python docker image work" or "make homebrew python work", for which logic equivalent to VIRTUAL_ENV=$(python -c "import sys; print(sys.prefix)") works fine, modulo the easy to add extra Windows case. I spent some time looking at distro specific sysconfig stuff or framework builds or whatever else and frankly supporting those base environments looks horrible (or at least beyond my amateur interest. Also if PEP 668 is used widely, maybe easy to declare out of scope)

[wpk-nist-gov]

I'm hoping a solution to this would solve an edge case I'm running into. I often have a conda environment with nox that then creates virtualenvs. This leads to both CONDA_PREFIX and VIRTUAL_ENV variables being set. With pip, this is easy to work around using path/to/python -m pip to install packages in the correct environment (so #1632, which I think is intimately connected to this issue, would solve this). But having a flag like --python-executable or --prefix would also solve this, as long as passing such a flag overrides CONDA_PREFIX and VIRTUAL_ENV. This would hopefully help wntrblm/nox#762 as well.

[charliermarsh]

I’m planning to work on this next week. I’ll post here with a concrete proposal once I’ve had time to internalize the comments. (Probably something like a --python flag alongside a --system flag but TBD.)

[charliermarsh]

As far as I can tell from reading the linked Poetry source, apart from adding the arguments to the CLI etc., the main thing that would need to change to support arbitrary Pythons (outside of a virtual environment) is that we'd need to call sysconfig.get_paths() to get the various environment paths, rather than constructing them ourselves -- since the exact structure can vary for non-virtualenvs. Is there any reason this wouldn't work?

[charliermarsh]

Definitely some things to learn from in the Poetry source: python-poetry/poetry@e8f259a

[fruch]

sharing one use case of using --prefix and --root,

inside a multi stages docker build, one stage is building/collecting all packages and install them into a /build directory
and other stages just copy the content out of into their /usr/local folder.

meanwhile I've worked around using those pip flags, by VIRTUAL_ENV=/user/local, and copy all of /user/local between the layers.

[RonNabuurs]

sharing one use case of using --prefix and --root,

inside a multi stages docker build, one stage is building/collecting all packages and install them into a /build directory and other stages just copy the content out of into their /usr/local folder.

meanwhile I've worked around using those pip flags, by VIRTUAL_ENV=/user/local, and copy all of /user/local between the layers.

This is the exact use case I also have

[fruch]

@charliermarsh I don't think the --python solved the equivalent pip install --root case, you want us to open a new issue for that ?

[strickvl]

@fruch there's also now a new --system flag which handles what you need, I think.

[Ox0400]

uv pip install --system uv