Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Second push of changes #5391

Merged
merged 37 commits into from
Aug 29, 2019
Merged

Conversation

richardliaw
Copy link
Contributor

@richardliaw richardliaw commented Aug 7, 2019

What do these changes do?

Related issue number

Linter

  • I've run scripts/format.sh to lint the changes in this PR.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16081/
Test PASSed.

@richardliaw richardliaw changed the title [docs] Second push of changes [wip][docs] Second push of changes Aug 8, 2019
@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16177/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16432/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16431/
Test PASSed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16452/
Test FAILed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16453/
Test PASSed.

@richardliaw richardliaw changed the title [wip][docs] Second push of changes [docs] Second push of changes Aug 22, 2019
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16469/
Test FAILed.

Copy link
Collaborator

@robertnishihara robertnishihara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Overall looks very cool.

It'll take a while to review everything here, so if you prefer to merge it and iterate then we can do that as well.

- `Tune`_: Hyperparameter Optimization Framework
- `RLlib`_: Scalable Reinforcement Learning
- `Distributed Training <http://ray.readthedocs.io/en/latest/distributed_training.html>`__
.. code-block:: python
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add some additional line above this one like "Execute Python functions in parallel."


.. _`installation documentation`: http://ray.readthedocs.io/en/latest/installation.html

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want these links to the papers? If we keep them, should we add the other relevant papers, e.g., RLlib?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should yeah (currently, they can be more informative than the rest of the documentation!)

doc/source/installation.rst Show resolved Hide resolved

.. code-block:: bash

pip install git+https://github.com/ray-project/ray.git#subdirectory=python
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line doesn't quite work because we still need bazel to be installed first. @mitar has pointed this out.

# Note that inside the remote function, the actual argument is provided.
return len(y)

argument = [1, 2, 3, 4]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer accurate. We put them in the object store if ray._raylet. returns False and otherwise we pickle them and inline them in the task spec. In this case, the list will be inlined in the task spec. This happens whenever the object consists of primitive types and/or numpy arrays and in aggregate is smaller than about 10KB ish.

Of course check_simple_value is not part of the Ray API.

# This implicitly places `argument` into the object store.
remote_function.remote(argument)

In the single node setting, all Ray workers can read the same object in the object store without copying (zero-copy reads). Once an object is placed in the object store, it is immutable. Any writes to the read-only object will result in a copy into the local process memory.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite true, writes are more likely to just raise an exception saying that the object is read only (for numpy arrays). For non numpy arrays, we actually do create a copy.

cc @pcmoritz

doc/source/object-store.rst Outdated Show resolved Hide resolved
doc/source/object-store.rst Outdated Show resolved Hide resolved
doc/source/object-store.rst Outdated Show resolved Hide resolved
@richardliaw richardliaw requested a review from ericl August 27, 2019 00:16
README.rst Outdated

`Tune`_ is a library for hyperparameter tuning at any scale.

- Launch a multi-node distributed asynchronous hyperparameter sweep in less than 10 lines of code.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Launch a multi-node distributed asynchronous hyperparameter sweep in less than 10 lines of code.
- Launch a multi-node distributed hyperparameter sweep in less than 10 lines of code.

- Whenever possible, use numpy arrays for maximum performance.


Object Store (Plasma)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved to the top of the doc, and the serialization stuff to the bottom (serialization is just not that important for most users.)

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16548/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16545/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16555/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16553/
Test PASSed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16561/
Test FAILed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16579/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16584/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16585/
Test PASSed.

@richardliaw richardliaw merged commit 411f30c into ray-project:master Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants