Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add action space to model #4210

Merged
merged 159 commits into from
Mar 10, 2019
Merged

Add action space to model #4210

merged 159 commits into from
Mar 10, 2019

Conversation

stefanpantic
Copy link
Contributor

What do these changes do?

  • Added action space to TensorFlow model class
  • Model.get_model now takes an aditional argument action_space which is the target gym env action space.

Aleksandar Milovanović and others added 30 commits January 23, 2019 17:59
made some changed and hacks so that our impala model is working (at l…
40fe041 
…east good enough)
added info whether env is started on local or remote evaluator
101bf50
fixed env knowing its worker_index and vector_index
5227eb3
added kl divergence and simplified MultiCategorical distribution
1fdab4a
Fixed variable name
ae49223
created remote environments
88f1e4e
Merge branch 'master' of https://gitlab.com/wingman-ai/ray
46203ec
improved handling of opening of external env
8280908
improved sample/train throughput logging
290d1f6
removing assert that remote evaluators exist from optimizer - this is…
ecd54f5 
… breaking rollout
made wingman changes
363e170
made wingman changes
7ad7b4c
abstracted away to discrete
f29e368
Renamed a method for consistency
2e22a06
Add TODO
60294ee
@stefanpantic
Add vector actions to IMPALA (#1)
a88f0df 
* Started unifying IMPALA Vtrace implementations so that they are action space agnostic

* Made VTrace work with both vector and discrete actions

* Cleaned up code

* Misc changes

* Removed tf.squeeze

* Reformat long line

* Changed condition

* Removed #noinspections

* Removed .pep8 config

* Changed model and behaviour dist

* Removed condition

* Changed log condition

* Changed ' to "

* Moved unstack outside of distribution for consistency
@stefanpantic
MultiDiscrete support (#2)
cd19654 
* Add support for gym.spaces.multi_discrete.MultiDiscrete action space to IMPALA

* Fixed shape error

* Removed output_hidden_shape

* Removed _is_multi_discrete boolean

* Remove action dtype

* Moved line outside of if
@stefanpantic
Merge branch 'master' of https://github.com/ray-project/ray
c68bec5 
# Conflicts:
#	python/ray/rllib/agents/agent.py
removed obsolete vtrace
2ed06cf
beatutifies changes a bit
15c8c27
a bit of align for pull request
3fbd2da
@bjg2
Entangled env (#3)
7a4646f 
* added entangled env

* improved entangled env
added all our environment changes
242aeea
added stats changes to optimizer
fe92b90
impala changes
a6b1b7a
Merge branch 'master' into all-changes
4cf9e43
merged with entangled-env
b7a36a7
merged better with entangled-env (comment)
1ec3a05
merged with optimizer-stats
19ad456
merged with impala
689a653
@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12429/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12430/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12432/
Test FAILed.

ericl
ericl approved these changes Mar 1, 2019
View reviewed changes
Copy link
Contributor

@ericl ericl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I pushed a change to satisfy the linter (scripts/format.sh)

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12442/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12609/
Test FAILed.

@ericl
Copy link
Contributor

ericl commented Mar 6, 2019

jenkins retest this please

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12625/
Test FAILed.

@ericl
Copy link
Contributor

ericl commented Mar 7, 2019

Looks like you missed a spot:

Traceback (most recent call last):
File "/ray/python/ray/rllib/tests/../examples/custom_loss.py", line 96, in
"input_files": args.input_files,
File

@stefanpantic
Copy link
Contributor Author

@ericl Fixed

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12663/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12665/
Test FAILed.

@ericl ericl merged commit 36cbde6 into ray-project:master Mar 10, 2019
@nikola-j nikola-j deleted the add-action-space-to-model branch March 27, 2019 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericl ericl ericl approved these changes

Assignees

@ericl ericl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@stefanpantic @AmplabJenkins @ericl @bjg2 @pimpke