Add Ray to Cluster computing

[jmargeta]

What is this Python project?

Ray is a flexible, high-performance distributed execution framework.

• Achieves parallelism in Python with simple and consistent API.
• Particularly suited for machine learning (agnostic to the machine learning library of choice)
• Forms the base of Ray's own distributed libraries for deep and reinforcement learning, processing of Pandas dataframes, and hyper parameter search.
• Uses Plasma as its object store which allows to efficiently share large numpy arrays (or objects serializable with Apache Arrow) between the processes, avoiding unnecessary data copies and with only minimal deserialization

What's the difference between this Python project and similar ones?

• Less overhead and lower latency with bottom up scheduling than similar projects
• Actors - allow sharing mutable state between tasks
• Similar to Dask, for more details see a comparison here: Comparison to dask ray-project/ray#642

--

Anyone who agrees with this pull request could vote for it by adding a 👍 to it, and usually, the maintainer will merge it when votes reach 20.

[pcmoritz]

This looks great, shall we merge this?

[DMITRIY1988]

Нормальный Кент живет обычно ему спокойно а жизнь прилична а если деньги мне подкинет я рифму круче зафуфырю #1170

[robertnishihara]

@vinta, would you be willing to take a look at this? I think this would be a great addition to the cluster computing section.