Cannot install on non-x86_64 architectures due to Ray dependency

[felker]

I am using 2x IBM AC922 systems (including OLCF Summit) with POWER9 CPUs and V100 GPUs, and it would be great to deploy DeepHyper on those systems. Although the DeepHyper binary wheel on PyPI is compatible since it specifies arch=any, I cannot install it due to the dependencies.

Ray does not distribute any .whl on PyPI for arch=ppc64le, only manylinux1 and macOS. There is an inactive open issue about this: ray-project/ray#4309

As I noted in #20 (comment), building Ray from source appears to be impossible right now.

Is there a way to bypass this dependency if only the BalsamEvaluator will be used?

[felker]

Getting close to building Ray on ppc64le; see ray-project/ray#4309

[felker]

After wrestling with incompatibilities of various versions of Arrow (from IBM's Conda channel), Bazel (built from source), Boost/Boost rules for Bazel, and Ray on ppc64le, I had to make only two lines of changes to Ray 0.8.0 to build it from source: ray-project/ray#4309 (comment)

I also had to build PostgreSQL from source in order to install the Psycopg2 dependency, but that is straightforward.

I now have a 14M ray-0.8.0-cp36-cp36m-linux_ppc64le.whl working Ray wheel. I can provide it (and a Python 3.7 version) to you if you want to upload it to PyPI.

Also, I quickly wrote a Travis CI build for ppc64le architectures to the following dependency for building Ray: nelhage/rules_boost#161
It might be a good idea to use it to model a similar pipeline for building and deploying Deephyper wheels, especially if you don't have easy access to such an architecture. Although this would require another pipeline for building and deploying Ray wheels. What is the role of the https://github.com/deephyper/ray fork?

[Deathn0t]

Hi @felker ,

That's a very good news!

For now, I just made a Deephyper Travis config to check automatically commits on the develop/master branch, but not everyone is adding automatic tests afterward...

The deephyper/ray fork is just a little patch to run ray on Theta at ANL where I was having a problem of IP address recognition in one of the modules it was looking for IP addresses corresponding to an ethernet interface by doing ifconfig and looking for an interface with a name starting by e which doesn't exist on Theta.

Thank you very much for keeping us posted!

[Deathn0t]

Now, ray or mpi4py are optional and can be installed/used to the preference of the user (deephyper==0.4.0)