Can not use ray with google-cloud-bigquery with poetry due to grpcio version incompatibility

[anurag-sequoia]

What happened + What you expected to happen

Tried installing ray using poetry in a virtual environment that already has google-cloud-bigquery.

Poetry dependency resolver fails because ray depends on grpcio <=1.43.0

Using version ^2.0.0 for ray

Updating dependencies
Resolving dependencies... (9.5s)

Because no versions of ray match >2.0.0,<3.0.0
 and ray (2.0.0) depends on grpcio (>=1.42.0,<=1.43.0), ray (>=2.0.0,<3.0.0) requires grpcio (>=1.42.0,<=1.43.0).
And because google-cloud-bigquery (3.3.2) depends on grpcio (>=1.47.0,<2.0dev)
 and no versions of google-cloud-bigquery match >3.3.2,<4.0.0, ray (>=2.0.0,<3.0.0) is incompatible with google-cloud-bigquery (>=3.3.2,<4.0.0).
So, because training depends on both google-cloud-bigquery (^3.3.2) and ray (^2.0.0), version solving failed.

Mac m1 requires newer grpcio dependencies - googleapis/python-bigquery#1262

Ideally, ray should have worked with newer grpcio dependencies as well.

Versions / Dependencies

ray == 2.0.0
google-cloud-bigquery == 3.3.2
macos 11.6.6
python 3.10.6

Reproduction script

# Get a new poetry environment on python 3.10 first

poetry add google-cloud-bigquery "ray[air, tune]"

Issue Severity

High: It blocks me from completing my task.

[rickyyx]

Hey @anurag-sequoia , thanks for reporting it. We do have a PR that's addressing the grpcio dependencies: #28623

I believe that should resolve this for you?

[anurag-sequoia]

Yes, thanks!