How to use --find-links with build inputs

[yajo]

I've spent a couple of days on this, so I guess it might be useful sharing my findings in case some other is having the same problem.

The summary is that I was building an Odoo addons repo that needed acsone/setuptools-odoo#115 included into its build of setuptools-odoo (which now doesn't make any sense to worry about, because it's been merged and released already).

To achieve that, I was building setuptools-odoo locally using dream2nix. Then, in the addon derivation, I was adding this to let it find that build dependency (only important parts included):

{config, ...}: {
  deps = {nixpkgs, ...}: {
    inherit (nixpkgs) setuptools-odoo; # Also built with d2n and overlaid
  };
  config.pip.pipFlags = ["--find-links=${config.deps.setuptools-odoo.out.dist}"];
}

FWIW the folder produced by the dist output just contained a file named setuptools_odoo-3.1.13.post11+g190da53-py2.py3-none-any.whl.

However, when nix run .#the_addon_repo.lock, pip was always finding the setuptools-odoo-3.1.12 candidate from PyPI.

After many diagnoses, I ended up finding out that the main problem was that the repo was being packaged using a setup.py file, as recommended upstream. That's not a problem per se, but a side effect is that, when you try to build a package that doesn't contain a pyproject.toml file, pip acts in legacy mode and doesn't pass the --find-links option. Thus, it tries to find the package in PyPI only.

The fix was to just add a pyproject.toml like this to the project:

[build-system]
requires = ["setuptools-odoo"]
build-backend = "setuptools.build_meta:__legacy__"

This lets pip trigger the modern PEP 517 support and install those requirements following its own standard systems, and passing the --find-links option while doing so.

I feel like I'd have been able to skip all this rabbit hole if I had a way to pass environment variables to the fetchPipMetadata script. Customizing that package is quite hard AFAICS.

It would be nice if it had some env variables input that one could add. Something like this:

{
  # Option 1
  config.pip.pipEnv.PIP_FIND_LINKS = "${config.deps.setuptools-odoo.out.dist}";
  # Option 2
  config.pip.lock.fields.fetchPipMetadata.args.env.PIP_FIND_LINKS = "${config.deps.setuptools-odoo.out.dist}";
}

Probably the legacy system of dependency downloads when there's a setup.py file and not a pyproject.toml file still would use that one.

Just sharing in case tomorrow me or somebody else needs this info, and in case this triggers some improvements in dream2nix interface.

Thanks!

[DavHau]

Interesting. Thanks for that report. I learned something new about how pip works.

We already have 'pip.nativeBuildInputd' which adds dependencies to the locking script. We could add 'pip.env'.

Feel free to open a PR if you like.

[yajo]

I opened #699.