[RLlib; new API stack by default] Switch on new API stack by default for SAC and DQN. #47217

sven1977 · 2024-08-20T12:55:42Z

Switch on new API stack by default for SAC and DQN.

Why are these changes needed?

Related issue number

Checks

I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
I've run scripts/format.sh to lint the changes in this PR.
I've included any doc changes needed for https://docs.ray.io/en/master/.
- I've added any new APIs to the API Reference. For example, if I added a
  method in Tune, I've added it in doc/source/tune/api/ under the
  corresponding .rst file.
I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
Testing Strategy
- Unit tests
- Release tests
- This PR is not tested :(

Signed-off-by: sven1977 <[email protected]>

…ch_on_new_api_stack_by_default_for_sac_and_dqn Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/algorithms/dqn/tests/test_dqn.py # rllib/algorithms/ppo/tests/test_repro_ppo.py # rllib/algorithms/sac/sac.py # rllib/algorithms/sac/tests/test_rnnsac.py # rllib/algorithms/sac/tests/test_sac.py

Signed-off-by: sven1977 <[email protected]>

simonsays1980

LGTM.

simonsays1980 · 2024-09-23T12:16:36Z

rllib/algorithms/dqn/dqn.py

        }
+        # `.api_stack()`
+        self.api_stack(


Why not self.enable_rl_module_and_learner = True?

There is this weird logic about the hybrid API stack's exploration settings, requiring us to go through the actual method call. We can remove all this logic, once the hybrid API stack is deprecated (soon! :) ).

…ch_on_new_api_stack_by_default_for_sac_and_dqn

Signed-off-by: sven1977 <[email protected]>

…ch_on_new_api_stack_by_default_for_sac_and_dqn

Signed-off-by: sven1977 <[email protected]>

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

sven1977 added 2 commits August 20, 2024 14:35

wip

1b97844

Signed-off-by: sven1977 <[email protected]>

wip

4d2352b

Signed-off-by: sven1977 <[email protected]>

sven1977 requested review from ArturNiederfahrenhorst, simonsays1980 and a team as code owners August 20, 2024 12:55

can-anyscale approved these changes Aug 22, 2024

View reviewed changes

sven1977 assigned simonsays1980 Sep 21, 2024

sven1977 added rllib RLlib related issues rllib-newstack rllib-oldstack-cleanup Issues related to cleaning up classes, utilities on the old API stack labels Sep 21, 2024

sven1977 added 4 commits September 22, 2024 11:43

fix

15ca1cb

Signed-off-by: sven1977 <[email protected]>

fix

706ec70

Signed-off-by: sven1977 <[email protected]>

wip

e1fae2c

Signed-off-by: sven1977 <[email protected]>

wip

6f9d09f

Signed-off-by: sven1977 <[email protected]>

simonsays1980 approved these changes Sep 23, 2024

View reviewed changes

sven1977 added 2 commits September 23, 2024 15:16

Merge branch 'master' of https://github.com/ray-project/ray into swit…

a0fb060

…ch_on_new_api_stack_by_default_for_sac_and_dqn

fix

9674b45

Signed-off-by: sven1977 <[email protected]>

sven1977 enabled auto-merge (squash) September 23, 2024 13:20

github-actions bot disabled auto-merge September 23, 2024 13:21

github-actions bot added the go add ONLY when ready to merge, run all tests label Sep 23, 2024

sven1977 enabled auto-merge (squash) September 23, 2024 13:22

sven1977 added 3 commits September 23, 2024 20:47

wip

f4bbec2

Signed-off-by: sven1977 <[email protected]>

Merge branch 'master' of https://github.com/ray-project/ray into swit…

6da95f3

…ch_on_new_api_stack_by_default_for_sac_and_dqn

wip

42d3dae

Signed-off-by: sven1977 <[email protected]>

github-actions bot disabled auto-merge September 23, 2024 18:48

sven1977 added 2 commits September 24, 2024 10:27

wip

b4a9dfc

Signed-off-by: sven1977 <[email protected]>

wip

95d8bdc

Signed-off-by: sven1977 <[email protected]>

sven1977 requested a review from maxpumperla as a code owner September 24, 2024 10:38

wip

7488af8

Signed-off-by: sven1977 <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

f5cf3e7

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

52c81ae

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

426be89

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

9b41b6e

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

0c2cc34

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

21a8b2a

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

492066c

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

aa4bb87

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

d4b68b3

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

c1e010b

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

d1beed9

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

9ba70f1

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

bdea958

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

cb43f88

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

912ad2c

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

9cffacd

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

27985d4

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

ujjawal-khare pushed a commit to ujjawal-khare-27/ray that referenced this pull request Oct 15, 2024

[RLlib; new API stack by default] Switch on new API stack by default …

bcda013

…for SAC and DQN. (ray-project#47217) Signed-off-by: ujjawal-khare <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RLlib; new API stack by default] Switch on new API stack by default for SAC and DQN. #47217

[RLlib; new API stack by default] Switch on new API stack by default for SAC and DQN. #47217

sven1977 commented Aug 20, 2024 •

edited

Loading

simonsays1980 left a comment

simonsays1980 Sep 23, 2024

sven1977 Sep 23, 2024

[RLlib; new API stack by default] Switch on new API stack by default for SAC and DQN. #47217

[RLlib; new API stack by default] Switch on new API stack by default for SAC and DQN. #47217

Conversation

sven1977 commented Aug 20, 2024 • edited Loading

Why are these changes needed?

Related issue number

Checks

simonsays1980 left a comment

Choose a reason for hiding this comment

simonsays1980 Sep 23, 2024

Choose a reason for hiding this comment

sven1977 Sep 23, 2024

Choose a reason for hiding this comment

sven1977 commented Aug 20, 2024 •

edited

Loading