Skip to content

RayDP-0.3.0

Compare
Choose a tag to compare
@carsonwang carsonwang released this 04 Jun 05:51

RayDP 0.3.0 includes the following key changes:

  • Spark dynamic resource allocation support. This allows you to launch Spark external shuffle service on Ray and enable Spark dynamic resource allocation to maximize your resource utilization.
  • Spark-submit support. A command line utility bin/raydp-submit is provided for you to submit a scala/java/python Spark application to a Ray cluster.
  • MPI on Ray. This allows you to run MPI jobs on Ray. You can use this feature to construct pipelines like Spark + MPI on Ray.
  • Ray 1.3.0 support.