Install wheel based on platform

[chohner]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: OSX 10.14.6

• Poetry version: Poetry 0.12.17

• Link of a Gist with the contents of your pyproject.toml file: not helpful as local files would be necessary

Issue

Based on this documentation, the following should be possible:

[tool.poetry.dependencies]
...
my_package = [
    { platform = "linux", file = "./my_package-1.0.0-cp37-cp37m-linux_x86_64.whl"},
    { platform = "darwin", file = "./my_package-1.0.0-cp37-cp37m-macosx_10_13_x86_64.whl" },
]

Each dependency is correctly handled on its own (aka ignored/installed), but pip seems to always try to install the first dependency if ANY dependency inside the array matches. So the above construct works on linux but not on osx, exchanging the line order swaps the behavior.

I understand this might be annoying to reproduce since it requires two local wheels for different platforms.

[lambdaclan]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Arch Linux and Windows 10

• Poetry version: 1.0.0

Issue

First of all thank you for your hard work. Poetry has made the otherwise painful Python dependency management much easier to deal with.

It seems like I have the same issue as @chohner.

Following the latest documentation regarding multiple constraint dependencies and using environment markers to define different requirements as stated

The constraints must have different requirements (like python) otherwise it will cause an error when resolving dependencies.

does not seem to work.

Here is the excerpt from my pyproject.toml

[tool.poetry.dependencies]
python = "^2.7"
PySide2 = [
    {path = "devpi/PySide2-5.12.0a1.dev1528120648-cp27-cp27m-win_amd64.whl", markers = "sys_platform == 'win32'" },
    {path = "devpi/PySide2-5.13.2-5.13.2-cp27-cp27mu-manylinux1_x86_64.whl", markers = "sys_platform == 'linux2'"}
]

Regardless of the platform, always the top package gets selected.

Executing a poetry update using the pyproject.toml above on Linux causes an exception because the Windows wheel gets selected:

   1: selecting pyside2 (5.12.0a1.dev1528120648 devpi/PySide2-5.12.0a1.dev1528120648-cp27-cp27m-win_amd64.whl)

[EnvCommandError]
ERROR: PySide2-5.12.0a1.dev1528120648-cp27-cp27m-win_amd64.whl is not a supported wheel on this platform.

This leads to a couple of questions:

1. Is there a workaround to make this work without the need of changing the package order?
2. Is it possible (or should it be) to mix match properties during multiple constraint resolution?

For example

my-package = [
    {url="https://example.devpi.com", markers = "sys_platform == 'win32'"},
   {path = "devpi/my-package-manylinux1_x86_64.whl", markers = "sys_platform == 'linux2'"}
]

If Windows fetch the package from internal devpi, if Linux use the file.

Any advice or further insight would be highly appreciated.

[jonfreedman]

Is there any solution to this? If not I cannot use poetry...

[jonfreedman]

The issues seem to be resolved for me with 1.1.0rc1 - poetry.lock now has two file entries for the dependency, one for each env

[chohner]

thanks for checking, seems to be working for me too! closing this issue

[malcolmgreaves]

Not working when using poetry version 1.1.3. The platform="..." and markers = "sys_platform == '...'"specifiers are still ignored and only the first entry is installed :(

Does anyone have an example of a successful work-around?

This is the simple entry I have in my pyproject.toml for poetry.tool.dependencies:

torch-spline-conv = [
    { markers = "sys_platform == 'darwin'", url = "https://pytorch-geometric.com/whl/torch-1.5.0/torch_spline_conv-latest%2Bcpu-cp38-cp38-macosx_10_9_x86_64.whl"},
    { markers = "sys_platform == 'linux'", url = "https://pytorch-geometric.com/whl/torch-1.5.0/torch_spline_conv-latest%2Bcpu-cp38-cp38-linux_x86_64.whl"}
]

NOTE: This has equivalent behavior for platform = "darwin" / platform = "linux". Omitted for brevity.

In my poetry.lock I have these entires:

[package.source]
type = "url"
url = "https://pytorch-geometric.com/whl/torch-1.5.0/torch_sparse-latest%2Bcpu-cp38-cp38-macosx_10_9_x86_64.whl"
[[package]]
name = "torch-spline-conv"
version = "1.2.0"
description = "Implementation of the Spline-Based Convolution Operator of SplineCNN in PyTorch"
category = "main"
optional = false
python-versions = ">=3.6"
...
torch-spline-conv = []

Regardless if I'm on OS X or Linux, poetry will always simply install the first thing, which is not correct behavior.

From this thread history, it appears as if there was a fix? If so, where can I get it?

[chohner]

I am on poetry 1.1.4 and

native_package = [
    { path = "./extern/native_package-cp37-cp37m-linux_x86_64.whl", platform = "linux" },
    { path = "./extern/native_package-cp37-cp37m-macosx_10_13_x86_64.whl", platform = "darwin" }
]

Installs fine on my mac (which used to fail trying to install the linux wheel) and linux CI.

[aatifsyed]

I'm also having difficulty with this:

poetry --version
Poetry version 1.1.4

[tool.poetry.dependencies]
# ...
# https://www.lfd.uci.edu/~gohlke/pythonlibs/
# https://stackoverflow.com/questions/54998028/how-do-i-install-pyaudio-on-python-3-7/54999645
pyaudio = [
    {path = "blobs/PyAudio-0.2.11-cp38-cp38-win_amd64.whl", platform = "windows" },
    {version = "0.2.11", platform = "linux" }
]

[hyliu1989]

The same thing happened to me, and I am using poetry 1.1.4. I have a path dependency that will include different extras in different platform. Different extras only work in their specific platform. I have a marker saying "sys_platform == 'win32'" and a second one saying "sys_platform == 'darwin'". It turns out the first one is chosen on MacOS and poetry tries to install the extra for win32, which failed. If I switch the order of the two constraints, poetry successfully install the constrained package on MacOS.

Could maintainer please reopen this issue? @abn

[pablo-olvin]

This problem was solved for me after updating to poetry 1.2.0b3. Behaviour is exactly what expected now.

[radoering]

Resolved via #5715.

[RRiva]

This bug still exists in Poetry 1.5.1. My issues is that tensorflow-io 0.32.0 provides wheels for macOS and linux, but not for Windows. Therefore, I wrote:

tensorflow-io = [
    {version="^0.32.0", markers="sys_platform == 'linux'"},
    {version="<=0.31.0", markers="sys_platform == 'win32'"},
]

Unfortunately, Poetry ignores the platform and tries to install 0.32.0 on Windows, which obviously fails.

[NeilGirdhar]

@RRiva Same thing happening to me with Jax

[cjuracek]

Still a problem in poetry v1.7.1. Attempting to maintain 2 separate versions of torch depending on platform:

[tool.poetry.dependencies]
...
torch = [
	{ platform = "darwin", version = "2.1.0" },
	{ platform = "linux", url = "https://download.pytorch.org/whl/cpu/torch-2.1.0%2Bcpu-cp38-cp38-linux_x86_64.whl#sha256=9e5cfd931a65b38d    222755a45dabb53b836be31bc620532bc66fee77e3ff67dc" }
]

On MacOS, it appears fine via poetry show torch:

 name         : torch                                                                      
 version      : 2.1.0                                                                      
 description  : Tensors and Dynamic neural networks in Python with strong GPU acceleration 

But fails to install the correct version on poetry install:

Installing torch (2.1.0+cpu https://download.pytorch.org/whl/cpu/torch-2.1.0%2Bcpu-cp38-cp38-linux_x86_64.whl#sha256=9e5cfd931a65b38d222755a45dabb53b836be31bc620532bc66fee77e3ff67dc): Failed

  RuntimeError

  Package https://download.pytorch.org/whl/cpu/torch-2.1.0%2Bcpu-cp38-cp38-linux_x86_64.whl#sha256=9e5cfd931a65b38d222755a45dabb53b836be31bc620532bc66fee77e3ff67dc cannot be installed in the current environment {'implementation_name': 'cpython', 'implementation_version': '3.8.15', 'os_name': 'posix', 'platform_machine': 'x86_64', 'platform_release': '21.6.0', 'platform_system': 'Darwin', 'platform_version': 'Darwin Kernel Version 21.6.0: Thu Nov  9 00:38:19 PST 2023; root:xnu-8020.240.18.705.10~1/RELEASE_X86_64', 'python_full_version': '3.8.15', 'platform_python_implementation': 'CPython', 'python_version': '3.8', 'sys_platform': 'darwin', 'version_info': [3, 8, 15, 'final', 0], 'interpreter_name': 'cp', 'interpreter_version': '38'}

This post is pretty much my exact problem. But as of now, I can't use the workaround proposed, as I need torch==2.1.0 on MacOS, which is not currently a supported wheel.

@radoering What is the preferred process here? Should I open a new issue, or keep everything within this issue?

[radoering]

In general it's probably better to open a new issue and link the related resolved issue if you think that a resolved issue is not resolved. (Most of the times the resolved issue is resolved and you are just experiencing a similar but different issue.)

In this specific case, a new issue is not required. You either want something like #7256 or (assuming that you expect that a wheel from PyPI is chosen for MacOS) you should force the source in the first constraint:

torch = [
	{ platform = "darwin", version = "2.1.0", source = "PyPI" },
	...
]

[cjuracek]

Thanks for the clarification - I'll note that for future issues.

Re: the proposed solution above, this now installs the correct torch on macOS, but fails to install the correct (CPU-based) version on Linux. Results of poetry show torch:

Actual

 name         : torch                                                                      
 version      : 2.1.0   

Expected

 name         : torch                                                                      
 version      : 2.1.0+cpu

Relevant pyproject.toml snippet: (with a custom source we specify via [[tool.poetry.source]])

torch = [
	{ platform = "darwin", version = "2.1.0", source = "artifactory" },
	{ platform = "linux", url = "https://download.pytorch.org/whl/cpu/torch-2.1.0%2Bcpu-cp38-cp38-linux_x86_64.whl#sha256=9e5cfd931a65b38d222755a45dabb53b836be31bc620532bc66fee77e3ff67dc" }
]

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.