[Tune][Doc] Restructure API reference #32311

justinvyu · 2023-02-08T09:13:41Z

Why are these changes needed?

This PR splits up long API refs into individual pages, one dedicated to each method/class. There are now 3 layers: an index page (ex: doc/tune/api/api.rst), a summary page for each group of APIs (ex: doc/tune/api/trainable.rst), and an individual page for each API (ex: doc/tune/api/docs/tune.Trainable.rst).

This PR is a followup to #31204, which made the same changes for Ray Data docs.

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: Justin Yu <[email protected]>

justinvyu · 2023-02-08T18:06:47Z

.gitignore

+/doc/source/*/api/doc/
+/doc/source/cluster/running-applications/job-submission/doc/


Can these be combined somehow? Special case for the newly added cluster autogenerated files.

@c21 should we conform to one structure? Though it does mess with backwards compatibility for this job submission page.

Yes I think we should conform to one structure (non-blocking for this PR). Let me take a look how easy the change is.

There was also a discussion about whether we should do reference/api or api/doc as the standard.

Also looks like I can use /doc/source/**/doc as wildcard for nested dirs.

Talked to @jjyao separately. We can use api/doc here. To rename to reference/api, we plan to have a separate discussion.

just keep in mind that api leads to shorter URLs, too. but consistency is more important

matthewdeng

Overall looks great, most of my comments are about changes we can make in the future 😄

matthewdeng · 2023-02-08T19:39:57Z

doc/source/tune/api/trainable.rst

@@ -240,7 +240,7 @@ Trainables can themselves be distributed. If your trainable function / class cre
 that also consume CPU / GPU resources, you will want to add more bundles to the :class:`PlacementGroupFactory`


It's already mentioned at the top of this file, but most/all of this page should exist in a user guide instead. We don't have to make the change in this PR though!

Agreed, I am thinking:

Move high level trainable stuff to key concepts trainable section

Move checkpointing guides to a new "Tune Checkpointing" user guide (the old one didn't have much to do with checkpointing, and a new one needs to be made)

Move the advanced resource allocation section to the tune resources user guide

Reuse actors section can probably go to tune lifecycle user guide.

doc/source/tune/api/trainable.rst

doc/source/tune/api/execution.rst

matthewdeng · 2023-02-08T19:55:32Z

doc/source/tune/api/suggestion.rst

+.. autosummary::
+    :toctree: doc/
+
+    flaml.BlendSearch


Same thing in master but we should fix this import.

I guess this is because doc requirements != Tune requirements? How should we go about fixing this?

matthewdeng · 2023-02-08T19:57:33Z

doc/source/tune/api/suggestion.rst


+    Searcher


nit: @c21 any suggested practice for separating out the constructor? I saw in Datasets APIs we explicitly do.

@matthewdeng - I think it's fine to keep it as it is here. I was following other system, such as Pandas - https://pandas.pydata.org/docs/reference/series.html .

matthewdeng · 2023-02-08T20:01:21Z

doc/source/tune/api/stoppers.rst

+.. autosummary::
+    :toctree: doc/
+
+    Stopper


Can we explicitly set this include _call_ as was done previously?

Currently this shows __init__ instead and feels a bit mismatched.

I don't think it's actually possible to include specific members with the autosummary. Best thing I could do was have it right below.

matthewdeng · 2023-02-08T20:08:56Z

doc/source/_toc.yml

@@ -282,7 +282,7 @@ parts:
              - file: tune/examples/exercises
                title: "Exercises"
          - file: tune/faq
-          - file: tune/api_docs/overview.rst
+          - file: tune/api/api.rst


Just checking, are you keeping Tune Internals and Tune Client API unchanged intentionally?

Yep, for Tune Internals, I wasn't sure how to list out all members/methods with their docstrings if I used autosummary. And since that section is not really meant for external users to search for and use as API calls, I decided to leave it fully expanded.

For Tune Client, there was only 1 class, and I think it is fine as is.

Ack, though we should probably move Callback and PlacementGroupFactory out of Tune Internals.

I think it would make sense to move PlacementGroupFactory directly into the Trainable Utilities section, next to with_resources.

Callback is a bit strange... it's a public tune API, which exposes internals about the trials, and is used by air.RunConfig... we should probably revisit this down the line.

Thoughts?

Moved Callback + PGF to better places.

c21

Clicked through several docs, and looks great to me, thanks @justinvyu!

c21 · 2023-02-09T00:13:33Z

Just FYI #32307 for Serve API is just merged, so rebasing to master should resvole the conflict here.

Signed-off-by: Justin Yu <[email protected]>

…/api_restructure

Signed-off-by: Justin Yu <[email protected]>

justinvyu · 2023-02-10T01:11:21Z

@maxpumperla Do you know why these errors are showing up?

/ray/doc/source/tune/api/doc/ray.tune.sklearn.TuneGridSearchCV.rst:48:<autosummary>:1: WARNING: term not in glossary: fit
/ray/doc/source/tune/api/doc/ray.tune.sklearn.TuneSearchCV.rst:48:<autosummary>:1: WARNING: term not in glossary: fit

The links all work fine, and fit is not missing in either of the generated autosummary/autofunctions.

See here and here.

Update: For now, I have changed this section back to what it was before (not showing any inherited members).

Signed-off-by: Justin Yu <[email protected]>

…/api_restructure

matthewdeng

@richardliaw can you merge?

* [Tune][Doc] Restructure API reference (#32311) * [Docs] Fix broken Tune links to overview and intergration (#32442) * Remove can_restore (exists on master but not releases/2.3.0) Signed-off-by: Justin Yu <[email protected]> --------- Signed-off-by: Justin Yu <[email protected]> Co-authored-by: Artur Niederfahrenhorst <[email protected]>

This PR splits up long API refs in AIR and Train into individual pages, one dedicated to each method/class. This PR is a followup to #31204 and #32311, which made the same changes for Ray Data/Tune docs. Signed-off-by: Justin Yu <[email protected]>

This PR splits up long API refs in AIR and Train into individual pages, one dedicated to each method/class. This PR is a followup to ray-project#31204 and ray-project#32311, which made the same changes for Ray Data/Tune docs. Signed-off-by: Justin Yu <[email protected]>

This PR splits up long API refs in AIR and Train into individual pages, one dedicated to each method/class. This PR is a followup to #31204 and #32311, which made the same changes for Ray Data/Tune docs. Signed-off-by: Justin Yu <[email protected]>

Signed-off-by: Edward Oakes <[email protected]>

This PR splits up long API refs in AIR and Train into individual pages, one dedicated to each method/class. This PR is a followup to ray-project#31204 and ray-project#32311, which made the same changes for Ray Data/Tune docs. Signed-off-by: Justin Yu <[email protected]> Signed-off-by: Edward Oakes <[email protected]>

This PR splits up long API refs in AIR and Train into individual pages, one dedicated to each method/class. This PR is a followup to ray-project#31204 and ray-project#32311, which made the same changes for Ray Data/Tune docs. Signed-off-by: Justin Yu <[email protected]> Signed-off-by: elliottower <[email protected]>

justinvyu added 11 commits February 7, 2023 17:11

Change directory from api_docs -> api

f72240c

Signed-off-by: Justin Yu <[email protected]>

gitignore for autogenerated api refs

4c54757

Signed-off-by: Justin Yu <[email protected]>

Restructure + update trainable api refs

d08c28f

Signed-off-by: Justin Yu <[email protected]>

update tune execution

a740de3

Signed-off-by: Justin Yu <[email protected]>

update search space + searchers

0e37dfb

Signed-off-by: Justin Yu <[email protected]>

Add TuneConfig to execution api ref section

994b554

Signed-off-by: Justin Yu <[email protected]>

Update schedulers

0a30fe5

Signed-off-by: Justin Yu <[email protected]>

update reporters, results, stoppers

d686153

Signed-off-by: Justin Yu <[email protected]>

update syncing, logging, sklearn

f8ea993

Signed-off-by: Justin Yu <[email protected]>

update integrations

1d1e9a3

Signed-off-by: Justin Yu <[email protected]>

Update sync config ref

745654d

Signed-off-by: Justin Yu <[email protected]>

justinvyu assigned matthewdeng Feb 8, 2023

justinvyu requested review from richardliaw, krfricke, xwjiang2010 and amogkam as code owners February 8, 2023 09:13

justinvyu assigned c21 Feb 8, 2023

justinvyu requested review from matthewdeng, Yard1, maxpumperla and a team as code owners February 8, 2023 09:13

justinvyu commented Feb 8, 2023

View reviewed changes

justinvyu added release-blocker P0 Issue that blocks the release P0 Issues that should be fixed in short order docs An issue or change related to documentation labels Feb 8, 2023

matthewdeng reviewed Feb 8, 2023

View reviewed changes

c21 reviewed Feb 9, 2023

View reviewed changes

justinvyu added 2 commits February 8, 2023 16:46

Fix invalid link refs

82b88f6

Signed-off-by: Justin Yu <[email protected]>

Strip parent modules for integrations section

93503c8

Signed-off-by: Justin Yu <[email protected]>

justinvyu added 4 commits February 8, 2023 18:35

Fix result grid section title

8c69afd

Signed-off-by: Justin Yu <[email protected]>

Set noindex on the air package ref for now

79c2cca

Signed-off-by: Justin Yu <[email protected]>

Fix optuna link

eb3bf64

Signed-off-by: Justin Yu <[email protected]>

Merge branch 'master' of https://github.com/ray-project/ray into tune…

032e7b4

…/api_restructure

justinvyu requested a review from gjoliver as a code owner February 9, 2023 07:44

justinvyu mentioned this pull request Feb 9, 2023

[AIR][Train][Doc] Restructure API reference #32360

Merged

8 tasks

Replace tune-sample-docs ref with tune-search-spaces

aaffb05

Signed-off-by: Justin Yu <[email protected]>

justinvyu added 2 commits February 9, 2023 17:26

Revert sklearn.rst change

31ece63

Signed-off-by: Justin Yu <[email protected]>

Merge branch 'master' of https://github.com/ray-project/ray into tune…

cfb0cf3

…/api_restructure

matthewdeng approved these changes Feb 10, 2023

View reviewed changes

richardliaw approved these changes Feb 10, 2023

View reviewed changes

richardliaw merged commit 613f4b0 into ray-project:master Feb 10, 2023

justinvyu added a commit to justinvyu/ray that referenced this pull request Feb 10, 2023

[Tune][Doc] Restructure API reference (ray-project#32311)

fa5cb8a

justinvyu mentioned this pull request Feb 10, 2023

[2.3.0][Tune][Doc] Restructure API reference #32449

Merged

7 tasks

matthewdeng mentioned this pull request Feb 13, 2023

[Tune] Add documentation for ray.tune.TuneConfig #30040

Closed

justinvyu mentioned this pull request Feb 14, 2023

[2.3.0][AIR][Train][Doc] Restructure API reference #32535

Merged

7 tasks

edoakes pushed a commit to edoakes/ray that referenced this pull request Mar 22, 2023

[Tune][Doc] Restructure API reference (ray-project#32311)

9627e8d

Signed-off-by: Edward Oakes <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tune][Doc] Restructure API reference #32311

[Tune][Doc] Restructure API reference #32311

justinvyu commented Feb 8, 2023

justinvyu Feb 8, 2023

matthewdeng Feb 8, 2023

c21 Feb 9, 2023

justinvyu Feb 9, 2023

justinvyu Feb 9, 2023

c21 Feb 9, 2023

maxpumperla Feb 9, 2023

matthewdeng left a comment

matthewdeng Feb 8, 2023

justinvyu Feb 9, 2023

matthewdeng Feb 8, 2023

justinvyu Feb 9, 2023

matthewdeng Feb 8, 2023

c21 Feb 9, 2023

matthewdeng Feb 8, 2023

justinvyu Feb 9, 2023

matthewdeng Feb 8, 2023

justinvyu Feb 8, 2023

matthewdeng Feb 8, 2023

justinvyu Feb 9, 2023

c21 left a comment

c21 commented Feb 9, 2023

justinvyu commented Feb 10, 2023 •

edited

Loading

matthewdeng left a comment

		/doc/source/*/api/doc/
		/doc/source/cluster/running-applications/job-submission/doc/

		@@ -240,7 +240,7 @@ Trainables can themselves be distributed. If your trainable function / class cre
		that also consume CPU / GPU resources, you will want to add more bundles to the :class:`PlacementGroupFactory`

[Tune][Doc] Restructure API reference #32311

[Tune][Doc] Restructure API reference #32311

Conversation

justinvyu commented Feb 8, 2023

Why are these changes needed?

Related issue number

Checks

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthewdeng left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

c21 left a comment

Choose a reason for hiding this comment

c21 commented Feb 9, 2023

justinvyu commented Feb 10, 2023 • edited Loading

matthewdeng left a comment

Choose a reason for hiding this comment

justinvyu commented Feb 10, 2023 •

edited

Loading