Implement pip's editable installation method for condaverse

[jezdez]

I wonder if, following the lack of resolution of the develop subcommand in conda-build, it would be worth implementing something similar using the capabilities of conda-pypi, basically providing a PyPI aware conda develop command.

[dholth]

I wonder if the tools provide straightforward dependency lists that could be fed into conda? Maybe not for the [build-system] requires? I see that python -m build has an --installer option... it would also be possible to build the ephemeral PEP "editable wheel" after installing build-system requirements, look at it and make some decisions.

[dholth]

I don't see why "conda-build develop" shouldn't be updated to use pep517.

There isn't much to it, but it uses the "add a .pth file to $CONDA_PREFIX/lib/python3.x/site-packages/conda.pth" mechanism.

PEP 517 is different. Instead of installing a .pth file, an ephemeral wheel is used which usually contains a .pth file, and will also contain dist-info metadata for that package. Uninstall is the same as uninstalling any wheel.

I would implement it with "no build isolation" so that everything was installed in the current conda environment.

We might copy https://github.com/pypa/pip/blob/HEAD/src/pip/_internal/pyproject.py, pip's pyproject.toml parser to get the build-system requirements, and conda-install those.

After those are installed it's possible to call https://pyproject-hooks.readthedocs.io/en/latest/pyproject_hooks.html#pyproject_hooks.BuildBackendHookCaller.build_editable (and possibly its prerequisite *_for_build_editable hooks, which many packages do not need) to get a wheel which, when installed, effects the editable install.

We would use https://packaging.pypa.io/en/stable/metadata.html#packaging.metadata.Metadata.requires_python to get the requirements and try to "conda install" them.

We could then delegate the rest of the install to "pip --no-deps".

So the same as pip except we make sure conda installs the dependencies in the places where they would otherwise have been installed with pip.

Possibly with some exceptions e.g. when there is a git URL https://peps.python.org/pep-0508/#examples

[jaimergp]

Have you tried running conda pip install -e .? What would happen in that case? We might need to special case the package argument for editable installs (so it doesn't use a potentially available conda counterpart) and local paths, but I think the current logic is not too far from implementing the necessary steps.

[jaimergp]

Ah, heh, the -e argument is not implemented. Let me look into it.

[dholth]

We should be able to add -e (or any pip argument) to the pip: section of an environment file but it would not use conda to install the dependencies.

[dholth]

pypa build does the isolated enviroment trick for wheel builds, does not support editable wheels (they recommend using the lower level project, which leaves the task of installing the build system and possibly creating an isolated environment undone) https://github.com/pypa/build/blob/main/src/build/env.py#L86-L90

Its version of parsing build-system

[jaimergp]

See #36 for the bare minimum changes needed. It should do the following:

• Run a pip dry-run with the editable package
• Parse the JSON and obtain the packages that would have been installed. Detect which ones are editable
• Map non-editable deps to conda-forge when available
• Install conda packages
• Install non-editable pypi packages
• Install editable pypi packages

[dholth]

See conda/conda-build#5380 for the (extremely rough) approach I would take.

[dholth]

See also https://github.com/pypa/installer to install a wheel without bothering with dependencies.