Use python_site_packages_path when specified

[jjhelmus]

I'm putting this up as an example and for discussion. More work is needed for a complete implementation, especially given that this one breaks backwards compatibility in various function.

Description

When the python package in an environment specifies a python_site_packages_path in its repodata entry, use that path to install noarch: python packages.

Note that this currently only works with the classic solver as libmamba/conda-libmamba-solver does not yet understand this field.

Close #14053.

Checklist - did you ...

• Add a file to the news directory (using the template) for the next release's release notes?
• Add / update necessary tests?
• Add / update outdated documentation?

[jjhelmus]

An example of this:

❯ conda create -n example -c jjhelmus/label/sp_path -c conda-forge python=3.99.90 imagesize --solver classic --yes
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /Users/jhelmus/workspace/conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/example

  added / updated specs:
    - imagesize
    - python=3.99.90


The following NEW packages will be INSTALLED:

  imagesize          conda-forge/noarch::imagesize-1.4.1-pyhd8ed1ab_0 
  python             jjhelmus/label/sp_path/noarch::python-3.99.90-0 



Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate example
#
# To deactivate an active environment, use
#
#     $ conda deactivate

❯ tree /Users/jhelmus/workspace/conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/example
/Users/jhelmus/workspace/conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/example
├── conda-meta
│   ├── history
│   ├── imagesize-1.4.1-pyhd8ed1ab_0.json
│   └── python-3.99.90-0.json
└── sample_path
    └── from_the_file
        └── index_json
            ├── imagesize
            │   ├── __init__.py
            │   └── imagesize.py
            └── imagesize-1.4.1.dist-info
                ├── INSTALLER
                ├── LICENSE.rst
                ├── METADATA
                ├── RECORD
                ├── REQUESTED
                ├── WHEEL
                ├── direct_url.json
                └── top_level.txt

7 directories, 13 files

The python package used here was created with conda-build that includes the change from conda/conda-build#5502.

[codspeed-hq]

CodSpeed Performance Report

Merging #14256 will not alter performance

_{Comparing jjhelmus:sp_dir (adbb7b9) with main (979716c)}

Summary

✅ 21 untouched benchmarks

[jjhelmus]

In ea89bc0 I added unit test to validate the noarch: python packages get installed into the path specified by the python_site_packages_path field. This uses one of the sample, empty packages made in conda/conda-build#5502 which is in my channel on anaconda.org. I'd be in favor or a more robust solution for testing if there are any suggestions.

[jezdez]

@jjhelmus You're right, we're trying to move away from relying on hosted pacakges, and have added test_recipes_channel pytest fixture (which is used in a number of places) that points the tests to the tests/test-recipes directory, in which we basically store slimmed down variants of packages for testing purposes, that we converted in #12879.

Could you add a custom python package to the directory and rely on it in your test?

cc @kenodegard since you may thoughts on this one

[jjhelmus]

Could you add a custom python package to the directory and rely on it in your test?

This is exactly what I was looking for. I'll update the PR to use this fixture and add the necessary packages. Thanks much.

[jjhelmus]

Test was updated to use the test_recipes_channel pytest fixture with two embedded packages.

[jjhelmus]

The single failing test, FAILED tests/core/test_solve.py::test_solve_1[libmamba] looks unrelated to this change.

[jjhelmus]

In order for this field to be usable with the libmamba solver, libmamba-conda-solver needs to pass this field, see conda/conda-libmamba-solver#560. This in turn requires mamba to support this field (mamba-org/mamba#3558) which mamba-org/mamba#3579 is a start of.