[RLlib] Enable Bandits to work in batches mode(s) (vector envs + multiple workers + train_batch_sizes > 1). #22465
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gjoliver
approved these changes
Feb 17, 2022
| self.arm.partial_fit(x[:, action_id], y) | ||
| for i, arm in enumerate(arms): | ||
| action_id = arm.item() | ||
| self.arm.partial_fit(x[[i], action_id], y[[i]]) |
There was a problem hiding this comment.
maybe we can comment here for ParametricLinearModel, x are all the potential arms, arms tells us which one was pulled instead. and the features pointed at by arms should be used to train the actual model "arm" ...
I know it's twisted, maybe it's better left un-explained ...
rkooo567
added a commit
to rkooo567/ray
that referenced
this pull request
Feb 18, 2022
…s + multiple workers + train_batch_sizes > 1). (ray-project#22465)" This reverts commit c58cd90.
6 tasks
simonsays1980
pushed a commit
to simonsays1980/ray
that referenced
this pull request
Feb 27, 2022
…iple workers + train_batch_sizes > 1). (ray-project#22465)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable Bandits to work in batches mode(s) (vector envs + multiple workers + train_batch_sizes > 1).
Why are these changes needed?
Related issue number
Checks
scripts/format.shto lint the changes in this PR.