Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Doc] Workspace template examples #32802

Merged
merged 81 commits into from
Mar 20, 2023
Merged

[Doc] Workspace template examples #32802

merged 81 commits into from
Mar 20, 2023

Conversation

justinvyu
Copy link
Contributor

@justinvyu justinvyu commented Feb 24, 2023 •
edited
Loading

Why are these changes needed?

This PR adds an initial set of examples that can be used as a template for users to get started with certain workloads.

This includes:

  1. Batch prediction w/ AIR
  2. Many model training w/ AIR
  3. Serving a stable diffusion model

For each example, we include:

  • Jupyter notebook
  • (Optional) requirements.txt for additional dependencies
  • Entry in templates.yaml
  • Entry in release tests yaml

See the contributing guide to see the steps needed to add a new template.

TODOs

  • Finish READMEs
  • Create hello world vs. production versions for all examples. Currently, there's just one version of each.
  • Add release tests

To add in a follow-up PR

  • Full Ray cluster launcher support
    • Guide on how to run these templates on OSS cluster
    • Show these templates somewhere in OSS docs
    • Create an script to convert OSS cluster config -> Anyscale configs

Related issue number

N/A

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

justinvyu and others added 25 commits February 16, 2023 11:15
@justinvyu
Check in example directories
fb94c45 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add example code to test
636d95b 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Fix batch inference + MMT examples to run properly
ebde3ec 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add working hello world version of serving stable diffusion
e0cc6f1 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Fix batch prediction example to use numpy
32d50eb 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Update README.md
3d45c13 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add example image
bd1072e 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Merge branch 'doc/example_templates' of https://github.com/justinvyu/ray
144081e 
 into doc/example_templates
@justinvyu
Add requirements.txt to stable diffusion example
4ad8594 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add readme skeletons
979663c 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Fix requirements (pin numpy + tensorboard)
53bde79 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Try out runtime env for dependencies
ee68212 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into doc/…
32f1aaa 
…example_templates
@justinvyu
Working serve example with runtime env
8f22df6 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into doc/…
f10ac4b 
…example_templates
@justinvyu
Load from requirements.txt file
73e0076 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add cluster env, compute config, templates.yaml for hello world examples
e136726 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Add 'large scale' stable diffusion serving example
02ded0c 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Reorganize configs to reduce duplication
36b2254 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Update mmt example with larger version
3643274 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Update batch inference example with image workload + large scale version
db4b688 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Merge branch 'master' of https://github.com/ray-project/ray into doc/…
4f1babd 
…example_templates
@justinvyu
Fix lint
3a677ac 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Update templates.yaml
242fcdb 
Signed-off-by: Justin Yu <[email protected]>
@justinvyu
Make batch inference work for small scale
26f1f4e 
Signed-off-by: Justin Yu <[email protected]>
ericl
ericl requested changes Mar 10, 2023
View reviewed changes
Copy link
Contributor

@ericl ericl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's actually do two things for now:

  1. Don't use BatchPredictor or Checkpoint APIs.
  2. Instead, use the Dataset ActorPool APIs and callable classes.

We are having some internal discussions on possibly deprecating / unifying BatchPredictor: #32929

@ericl ericl added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Mar 10, 2023
@ericl ericl self-assigned this Mar 10, 2023
@richardliaw richardliaw requested a review from ericl March 16, 2023 17:04
richardliaw
richardliaw reviewed Mar 16, 2023
View reviewed changes
doc/source/conf.py Outdated
@@ -160,6 +160,7 @@
"_build",
"source/workflows/api/doc/ray.workflow.*",
"source/serve/api/doc/ray.serve.*",
"source/templates",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe source/templates/*?

ericl
ericl reviewed Mar 16, 2023
View reviewed changes
doc/source/templates/01_batch_inference/batch_inference.ipynb
"NUM_WORKERS: int = 1\n",
"\n",
"USE_GPU: bool = True\n",
"NUM_GPUS_PER_WORKER: float = 1\n"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cell is repeated twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how I'm getting around loading a small vs. large version of the template. On product side, we can run some post init command to process the notebook:

  • Filter out all notebook cells tagged as large. Vice versa for large scale examples. This way, the configurations get set to use the small example defaults.
jupyter nbconvert --TagRemovePreprocessor.remove_input_tags='large'
      --to notebook --output batch_inference.ipynb batch_inference.ipynb

How does that sound @ericl?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we comment these cells like # This is the large case, etc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sounds good!

ericl
ericl reviewed Mar 16, 2023
View reviewed changes
doc/source/templates/01_batch_inference/batch_inference.ipynb Outdated
"metadata": {},
"outputs": [],
"source": [
"preds = predictions.fully_executed()\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"preds = predictions.fully_executed()\n",
"preds = predictions.cache()\n",

The fully executed call is deprecated, use cache() instead.

@ericl ericl added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Mar 16, 2023
@justinvyu justinvyu added tests-ok The tagger certifies test failures are unrelated and assumes personal liability. and removed @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. labels Mar 19, 2023
@richardliaw richardliaw changed the title [Doc][Anyscale] Workspace template examples [Doc] Workspace template examples Mar 20, 2023
@richardliaw richardliaw merged commit ed2982f into ray-project:master Mar 20, 2023
edoakes pushed a commit to edoakes/ray that referenced this pull request Mar 22, 2023
@justinvyu @edoakes
[Doc] Workspace template examples (ray-project#32802)
dde6f1b 
Signed-off-by: Edward Oakes <[email protected]>
clarng pushed a commit to clarng/ray that referenced this pull request Mar 23, 2023
@justinvyu @clarng
[Doc] Workspace template examples (ray-project#32802)
3117186
chaowanggg pushed a commit to chaowanggg/ray-dev that referenced this pull request Apr 4, 2023
@justinvyu @chaowanggg
[Doc] Workspace template examples (ray-project#32802)
0cfe1fd 
Signed-off-by: chaowang <[email protected]>
elliottower pushed a commit to elliottower/ray that referenced this pull request Apr 22, 2023
@justinvyu @elliottower
[Doc] Workspace template examples (ray-project#32802)
8668841 
Signed-off-by: elliottower <[email protected]>
ProjectsByJackHe pushed a commit to ProjectsByJackHe/ray that referenced this pull request May 4, 2023
@justinvyu @ProjectsByJackHe
[Doc] Workspace template examples (ray-project#32802)
7bac030 
Signed-off-by: Jack He <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericl ericl ericl approved these changes

@richardliaw richardliaw richardliaw approved these changes

Assignees

@ericl ericl

@matthewdeng matthewdeng

@richardliaw richardliaw

Labels
tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@justinvyu @ericl @richardliaw @matthewdeng