[Rllib] Add timeout to filter synchronization #25959
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ArturNiederfahrenhorst
requested review from
sven1977,
gjoliver,
avnishn,
smorad,
maxpumperla,
kouroshHakha and
krfricke
as code owners
|
oh, I was gonna use connectors to implement this filters ... seems like extra complexity if they need to be synced. |
sven1977
reviewed
Jun 23, 2022
sven1977
reviewed
Jun 23, 2022
sven1977
reviewed
Jun 23, 2022
sven1977
approved these changes
Jun 23, 2022
|
Hey @gjoliver , yeah filters may still have some features (synching) that we don't support with connectors yet. But I'm not too sure it's really that important. Even if you use something like MeanStdFilter and workers don't sync, you'd probably still be able to learn properly given that the distribution of states that each worker's trajectory covers is somewhat uniform between the workers. |
truelegion47
pushed a commit
to truelegion47/ray
that referenced
this pull request
Jun 30, 2022
* master: (35 commits) [tune/structure] Refactor `suggest` into `search` package (ray-project#26074) Add back ray.state in deprecation wrapper; print stack trace on warning (ray-project#26086) Enable isort for base directory (ray-project#26085) [AIR] Add __init__.py to ray.air.callbacks (ray-project#26088) [Serve] [Docs] Create end-to-end documentation example for Serve REST API and CLI (ray-project#25936) [AIR] Remove unnecessary pandas from examples (ray-project#26009) [Datasets] [Hotfix] Update `ds.to_pandas()` limit error to reflect current limit API (ray-project#26081) [Serve] [Docs] Add Serve REST API Schema to Serve API Docs (ray-project#25786) [Core][Doc] remove cython section from advanced doc. ray-project#26062 [Core] Fix check failure from incorrect death cause (ray-project#26007) [hotfix] Fix linkcheck (ray-project#26070) [RLlib] Add timeout to filter synchronization. (ray-project#25959) [tune/structure] Introduce logger package (ray-project#26049) [RLlib] introduce serialization for our custom gym space types. (ray-project#25923) Fix unit test test_check_env.py and est_check_multi_agent.py. (ray-project#25993) [RLlib] Make QMix use the ReplayBufferAPI (ray-project#25560) [CI] deflake test_multi_node_3 by increasing its timeout [CI] Use BUILDKITE_JOB_ID for better navigation for flaky tracker (ray-project#26021) [AIR/Docs] Improve user guide gallery (ray-project#26016) 🎨 Update type annotations to include options in `ray.remote()` (ray-project#25999) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
The filters that Rollout Workers apply to metrics are updated regularely as part of the training_step().
This operation includes a ray.get() that is lacking a timeout.
Without knowing the specifics of why this times out on my local machine when running IMPALA, I propose to set a timeout here so that we do not get stuck on this unnoticed.
Checks
scripts/format.shto lint the changes in this PR.