Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Discussion 3001: Fix comment on internal state shape (must be [B x S=state dim]). #17341

Merged
merged 4 commits into from
Jul 28, 2021
Merged

[RLlib] Discussion 3001: Fix comment on internal state shape (must be [B x S=state dim]). #17341

merged 4 commits into from
Jul 28, 2021

Conversation

sven1977
Copy link
Contributor

@sven1977 sven1977 commented Jul 26, 2021 •
edited
Loading

[RLlib] Fix comment on internal state shape (must be [B x S=state dim]).

Also see this discussion here:
https://discuss.ray.io/t/understanding-state-batches-in-compute-actions/3001/3

Why are these changes needed?

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@sven1977 sven1977 added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Jul 26, 2021
stefanbschneider
stefanbschneider approved these changes Jul 26, 2021
View reviewed changes
Copy link
Member

@stefanbschneider stefanbschneider left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I couldn't find any other places in the documentation where this would need to be changed.

Is there any other work to be done here?

@sven1977
Copy link
Contributor Author

Checked all the remaining code. Didn't find any more occurrences of batch and state-dim being reversed.

@sven1977 sven1977 merged commit 58da5c1 into ray-project:master Jul 28, 2021
stephanie-wang pushed a commit to stephanie-wang/ray that referenced this pull request Jul 31, 2021
@sven1977 @stephanie-wang
[RLlib] Discussion 3001: Fix comment on internal state shape (must be…
2bfc416 
… [B x S=state dim]). (ray-project#17341)
DmitriGekhtman pushed a commit that referenced this pull request Aug 3, 2021
@sasha-s @AmeerHajAli
@clay4megtr @edoakes @simon-mo @amogkam @richardliaw @sven1977 @Yard1 @Rohan138 @amavilla @jiaodong @juliusfrost @ericl @kk-55 @fishbone @matthewdeng
[autoscaler] Enforce per-node-type max workers (#17352)
5dbbaf7 
* Enforce per-node-type max workers

* type annonation

Co-authored-by: Ameer Haj Ali <[email protected]>

* cleanup. comments. type annotations

* additional type annotation

Co-authored-by: Ameer Haj Ali <[email protected]>

* additional cleanup. comments. type annotations

* _get_nodes_needed_for_request_resources to use FrozenSet

* comments

* whitespace

* [Placement Group] Fix resource index assignment between with bundle index and without bundle index pg (#17318)

* [serve] Add Ray API stability annotations (#17295)

* Support streaming output of runtime env setup to logger/driver (#17306)

* [SGD] v2 prototype: ``WorkerGroup`` implementation (#17330)

* wip

* formatting

* increase timeouts

* address comments

* comments

* fix

* address comments

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <[email protected]>

* Update python/ray/util/sgd/v2/worker_group.py

Co-authored-by: Richard Liaw <[email protected]>

* address comments

* formatting

* fix

* avoid race condition

Co-authored-by: Richard Liaw <[email protected]>

* [RLlib] Discussion 3001: Fix comment on internal state shape (must be [B x S=state dim]). (#17341)

* [autoscaler] GCP TPU VM autoscaler (#17278)

* [Rllib] set self._allow_unknown_config (#17335)

Co-authored-by: Sven Mika <[email protected]>

* [RLlib] Discussion 2294: Custom vector env example and fix. (#16083)

* [docs] Link broken in Tune's page (#17394) (#17407)

* [Serve] Fix response_model for class based view routes as well (#17376)

* [serve] Fix single deployment nightly test (#17368)

* [RLlib] SAC tuple observation space fix (#17356)

* Support schema on read for csv/json (#17354)

* [RLlib] New and changed version of parametric actions cartpole example + small suggested update in policy_client.py (#15664)

* [gcs] Fix GCS related issues: ByteSizeLong and redis connection (#17373)

* [runtime_env] Gracefully fail tasks when an environment fails to be set up (#17249)

* [docs] update docs with pip requirements (#17317)

* removed nodes_to_keep. cleanup

* formatting

* +comment

* treat max_workers=0 as 0 workers (as opposed to unlimited)

* fix wrong comment

* warning for inconsistent config

* terminate nodes with no matching node type right away

* quotes

* special handling for head node when enforcing max_workers per type. tests. cleanup

* cleanup comments and prints

* comments

* cleanup. removed special handling of head node.

* adding an eplicit non-None check in schedule_node_termination

* raise the exception

Co-authored-by: Ameer Haj Ali <[email protected]>

Co-authored-by: Ameer Haj Ali <[email protected]>
Co-authored-by: DK.Pino <[email protected]>
Co-authored-by: Edward Oakes <[email protected]>
Co-authored-by: Simon Mo <[email protected]>
Co-authored-by: Amog Kamsetty <[email protected]>
Co-authored-by: Richard Liaw <[email protected]>
Co-authored-by: Sven Mika <[email protected]>
Co-authored-by: Antoni Baum <[email protected]>
Co-authored-by: Rohan138 <[email protected]>
Co-authored-by: amavilla <[email protected]>
Co-authored-by: Jiao <[email protected]>
Co-authored-by: Julius Frost <[email protected]>
Co-authored-by: Eric Liang <[email protected]>
Co-authored-by: kk-55 <[email protected]>
Co-authored-by: Yi Cheng <[email protected]>
Co-authored-by: matthewdeng <[email protected]>
@sven1977 sven1977 deleted the fix_comment_on_internal_state_shape branch June 2, 2023 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stefanbschneider stefanbschneider stefanbschneider approved these changes

@michaelzhiluo michaelzhiluo Awaiting requested review from michaelzhiluo

Assignees

@richardliaw richardliaw

@michaelzhiluo michaelzhiluo

@stefanbschneider stefanbschneider

Labels
tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@sven1977 @stefanbschneider @richardliaw @michaelzhiluo