Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib] Add examples and docs for Catalog #33898

Merged
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
a0a3445
Add custom_action_distribution example
ArturNiederfahrenhorst Mar 30, 2023
8e30ab9
Merge branch 'master' into catalogexamplesanddocs
ArturNiederfahrenhorst Apr 3, 2023
7c5dc31
Added examples and svgs
ArturNiederfahrenhorst Apr 3, 2023
ded1db1
Add more text and include examples into new file 'rllib-catalogs.rst'
ArturNiederfahrenhorst Apr 4, 2023
dfcf353
wip
ArturNiederfahrenhorst Apr 4, 2023
e1da6a5
wip
ArturNiederfahrenhorst Apr 4, 2023
ac0912a
lynt
ArturNiederfahrenhorst Apr 4, 2023
b85d855
Avnish's comment about rlmodules rollout banner
ArturNiederfahrenhorst Apr 4, 2023
fb2c083
Update doc/source/rllib/key-concepts.rst
ArturNiederfahrenhorst Apr 9, 2023
b9518fc
Update doc/source/rllib/key-concepts.rst
ArturNiederfahrenhorst Apr 9, 2023
192b106
Update doc/source/rllib/key-concepts.rst
ArturNiederfahrenhorst Apr 9, 2023
da08bbe
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
2a7cb5e
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
1772715
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
a9b86c1
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
fdb2bcf
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
bd71be7
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
a91928b
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
5abb2b6
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
6d93736
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 9, 2023
5066830
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
342af33
More comments by Kourosh
ArturNiederfahrenhorst Apr 9, 2023
06af93e
More comments from Kourosh
ArturNiederfahrenhorst Apr 10, 2023
4eeb143
Merge branch 'master' into catalogexamplesanddocs
ArturNiederfahrenhorst Apr 10, 2023
f69b93b
Add references to catalogs doc
ArturNiederfahrenhorst Apr 10, 2023
8f3a57d
Merge master
ArturNiederfahrenhorst Apr 10, 2023
b722b20
Add Catalog links to rlmodule doc
ArturNiederfahrenhorst Apr 10, 2023
b2414cf
Add highlightings to keyswords
ArturNiederfahrenhorst Apr 10, 2023
65bebb5
more highlighting and fit topics to headings
ArturNiederfahrenhorst Apr 10, 2023
8025462
Fix list being off
ArturNiederfahrenhorst Apr 10, 2023
fae6c2b
Add package references
ArturNiederfahrenhorst Apr 10, 2023
07114e9
Revert change to catalog.py because this is a pure docs PR
ArturNiederfahrenhorst Apr 10, 2023
f90a043
Move examples
ArturNiederfahrenhorst Apr 10, 2023
607b132
Add flake8 noqa to catalog_guide
ArturNiederfahrenhorst Apr 10, 2023
d71c5a2
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
6260b00
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
1610b13
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
8fb152c
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
cbce31f
lynt
ArturNiederfahrenhorst Apr 10, 2023
0cca86b
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
41dc7b0
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
5d7608c
Update doc/source/rllib/package_ref/catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
deaaca5
Angelina's comment
ArturNiederfahrenhorst Apr 10, 2023
c80a4e9
Angelina's comment
ArturNiederfahrenhorst Apr 10, 2023
4a16d73
Angelina's comment
ArturNiederfahrenhorst Apr 10, 2023
5eb5f2f
Update doc/source/rllib/package_ref/catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
d07f219
Update doc/source/rllib/package_ref/catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
6d562ad
Update doc/source/rllib/package_ref/catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
766ac2f
Update doc/source/rllib/package_ref/catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
2a16afb
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
a819a05
Update doc/source/rllib/rllib-catalogs.rst
ArturNiederfahrenhorst Apr 10, 2023
a375bf3
Update doc/source/rllib/rllib-rlmodule.rst
ArturNiederfahrenhorst Apr 10, 2023
d1a3fe3
Update doc/source/rllib/user-guides.rst
ArturNiederfahrenhorst Apr 10, 2023
ac00433
Update rllib/examples/catalog/custom_action_distribution.py
ArturNiederfahrenhorst Apr 10, 2023
8454f7c
Update rllib/examples/catalog/custom_action_distribution.py
ArturNiederfahrenhorst Apr 10, 2023
b29b5bc
Make RLModule->RLModule changes
ArturNiederfahrenhorst Apr 10, 2023
4524cd4
Fix Documentation errors
ArturNiederfahrenhorst Apr 10, 2023
24e2650
Fix catalog guide
ArturNiederfahrenhorst Apr 11, 2023
048a01d
Merge branch 'master' into catalogexamplesanddocs
ArturNiederfahrenhorst Apr 11, 2023
92df6eb
lynt
ArturNiederfahrenhorst Apr 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions doc/source/_includes/rllib/rlmodules_rollout.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
..
.. admonition:: This doc is related to the RLModules API.

Disregard this doc if you do not explicitly want to use RLModules. The RLModules API is experimental and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could probably just say something to the effect of RLModules is an experimental API. I think this gets across the point overall. Its implied what experimental means.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide a complete sentence to replace this with?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RLModules api is currently in the experimental phase of its rollout.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm while we're at it, is this a whole separate doc page right here? If so, why?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an include. It's basically a banner that alerts users that they are looking at something experimental.
Angelina told me to make this basically the first thing people read - if it's not meant for John Doe, put it first.

CC @kouroshHakha to make you aware of this banner

only being rolled out across RLlib. It is not yet recommended for general until it is fully rolled out.
3 changes: 2 additions & 1 deletion doc/source/_toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ parts:
title: Ray RLlib
sections:
- file: rllib/rllib-training
- file: rllib/core-concepts
- file: rllib/key-concepts
- file: rllib/rllib-env
- file: rllib/rllib-algorithms
- file: rllib/user-guides
Expand All @@ -299,6 +299,7 @@ parts:
- file: rllib/rllib-sample-collection
- file: rllib/rllib-replay-buffers
- file: rllib/rllib-offline
- file: rllib/rllib-catalogs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put this under user-guide for now. It's not a key concept yet since it's experimental. We have to later re-structure the key-concepts page to have better navigation around topics.

- file: rllib/rllib-connector
- file: rllib/rllib-fault-tolerance
- file: rllib/rllib-dev
Expand Down
16 changes: 16 additions & 0 deletions doc/source/rllib/images/catalog/catalog_and_rlm_diagram.svg
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of the object behind "action space" is difficult to read. I think it's the "observation space"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. It's not meant to be readable. It's a bunch of things that are described above that go into the Catalog.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change this?

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading