Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Remove all rllib_contrib algo folders from algorithms/. #43656

Merged
merged 44 commits into from
Mar 15, 2024
Merged

[RLlib] Remove all rllib_contrib algo folders from algorithms/. #43656

merged 44 commits into from
Mar 15, 2024

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Mar 3, 2024

Remove all rllib_contrib algo folders from algorithms/.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@sven1977
wip
8277abd 
Signed-off-by: sven1977 <[email protected]>
@sven1977 sven1977 added rllib RLlib related issues rllib-contrib rllib-oldstack-cleanup Issues related to cleaning up classes, utilities on the old API stack labels Mar 3, 2024
@sven1977
fixes
735df2a 
Signed-off-by: sven1977 <[email protected]>
simonsays1980
simonsays1980 approved these changes Mar 4, 2024
View reviewed changes
Copy link
Collaborator

@simonsays1980 simonsays1980 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great cleanup.

python/ray/tests/test_client_library_integration.py
config["num_workers"] = 0
# Test with compression.
config["compress_observations"] = True
config = (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Finally this dependence vanishes and makes DQN Rainbow easier to integrate.

rllib/algorithms/dqn/dqn.py

logger = logging.getLogger(__name__)


class DQNConfig(SimpleQConfig):
class DQNConfig(AlgorithmConfig):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yess :)

rllib/algorithms/dqn/dqn.py
hiddens: Dense-layer setup for each the advantage branch and the value
branch
double_q: Whether to use double DQN.
n_step: N-step for Q-learning.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add here that random n-step sampling can be applied via tuple.

@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
79e47f5 
…ecate_rllib_contrib_algos
@sven1977
wip
36628d0 
Signed-off-by: sven1977 <[email protected]>
@sven1977
LINT
a255169 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
25c203a 
Signed-off-by: sven1977 <[email protected]>
@sven1977 sven1977 requested a review from a team as a code owner March 4, 2024 14:27
@sven1977
wip
cf5d90c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
be776e2 
…ecate_rllib_contrib_algos
@sven1977
wip
27b821a 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
294b8e2 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' into deprecate_rllib_contrib_algos
40a7c8e
@sven1977
wip
19eb90e 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge remote-tracking branch 'origin/deprecate_rllib_contrib_algos' i…
1791d8e 
…nto deprecate_rllib_contrib_algos
@sven1977
Merge branch 'master' into deprecate_rllib_contrib_algos
639d339
@sven1977
Merge branch 'master' into deprecate_rllib_contrib_algos
b614114
@sven1977
wip
ad4f237 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
5e41d56 
…ecate_rllib_contrib_algos
@sven1977
Merge remote-tracking branch 'origin/deprecate_rllib_contrib_algos' i…
a23cf2a 
…nto deprecate_rllib_contrib_algos
@sven1977
wip
60f9ede 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
4f3621b 
…ecate_rllib_contrib_algos
@sven1977
wip
872a0a9 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
c1c4025 
Signed-off-by: sven1977 <[email protected]>
@sven1977
merge
24fc4ed 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
c00d33f 
…ecate_rllib_contrib_algos

Signed-off-by: sven1977 <[email protected]>

# Conflicts:
#	rllib/algorithms/tests/test_worker_failures.py
@sven1977
LINT
f6da172 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
c7a03e4 
…ecate_rllib_contrib_algos
@sven1977
wip
a8d80cc 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
fb85f20 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
f275148 
…ecate_rllib_contrib_algos
@sven1977
wip
a23344c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
fee09e2 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
9b9ea79 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
0be5a2c 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
6d48c92 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
f5c1263 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
e8d564d 
…ecate_rllib_contrib_algos
@sven1977
wip
65f2f48 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
472ef2b 
Signed-off-by: sven1977 <[email protected]>
@sven1977
Merge branch 'master' of https://github.com/ray-project/ray into depr…
e8e5200 
…ecate_rllib_contrib_algos
@sven1977
wip
f55d1a2 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
908f164 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
ff14438 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
b670f15 
Signed-off-by: sven1977 <[email protected]>
@sven1977
wip
bbe7508 
Signed-off-by: sven1977 <[email protected]>
@sven1977 sven1977 merged commit 82e6c40 into ray-project:master Mar 15, 2024
9 checks passed
@sven1977 sven1977 deleted the deprecate_rllib_contrib_algos branch March 15, 2024 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simonsays1980 simonsays1980 simonsays1980 approved these changes

@avnishn avnishn Awaiting requested review from avnishn

@ArturNiederfahrenhorst ArturNiederfahrenhorst Awaiting requested review from ArturNiederfahrenhorst

@maxpumperla maxpumperla Awaiting requested review from maxpumperla maxpumperla is a code owner

@kouroshHakha kouroshHakha Awaiting requested review from kouroshHakha

Assignees

@simonsays1980 simonsays1980

Labels
rllib RLlib related issues rllib-oldstack-cleanup Issues related to cleaning up classes, utilities on the old API stack
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sven1977 @simonsays1980