[RLlib] Metrics do-over 04: New env rendering/video example script (through custom callbacks using MetricsLogger). #45073
Commits on Mar 13, 2024
Commits on Apr 2, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
Commits on Apr 3, 2024
Commits on Apr 4, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
Commits on Apr 6, 2024
Commits on Apr 8, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into clea…
…nup_examples_folder_03 # Conflicts: # rllib/examples/multi_agent/multi_agent_pendulum.py and wip Signed-off-by: sven1977 <[email protected]>
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
-
Merge branch 'master' of https://github.com/ray-project/ray into clea…
…nup_examples_folder_03
Commits on Apr 9, 2024
-
Apply suggestions from code review
Co-authored-by: angelinalg <[email protected]> Signed-off-by: Sven Mika <[email protected]>
-
Merge remote-tracking branch 'origin/cleanup_examples_folder_03' into…
… cleanup_examples_folder_03
-
Merge branch 'master' of https://github.com/ray-project/ray into clea…
…nup_examples_folder_03
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
Commits on Apr 15, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
Commits on Apr 16, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
Commits on Apr 17, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over
-
WandB logging of videos working!
Signed-off-by: sven1977 <[email protected]>
Commits on Apr 18, 2024
Commits on Apr 19, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_main_branch
Commits on Apr 20, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into comp…
…lete_metrics_and_stats_do_over Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/utils/test_utils.py
-
Merge remote-tracking branch 'origin/metrics_do_over_02_algo_and_ppo_…
…training_step' into metrics_do_over_02_algo_and_ppo_training_step
Commits on Apr 21, 2024
-
Signed-off-by: sven1977 <[email protected]>
-
Merge branches 'master' and 'master' of https://github.com/ray-projec…
…t/ray into metrics_do_over_02_algo_and_ppo_training_step
Commits on Apr 22, 2024
Commits on Apr 23, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_02_algo_and_ppo_training_step Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/algorithms/algorithm.py # rllib/algorithms/dqn/dqn.py
-
SAC Learning Pendulum (careful about the prio replay buffer setting, …
…they matter quite a lot!). capacity=100k alpha=1.0 beta=0.0 Signed-off-by: sven1977 <[email protected]>
-
SAC Learning Pendulum (after sharing its training_step code with DQN,…
… except noisey net) Signed-off-by: sven1977 <[email protected]>
Commits on Apr 24, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_02_algo_and_ppo_training_step
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_main_branch Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/algorithms/algorithm.py
-
Merge branch 'metrics_do_over_02_algo_and_ppo_training_step' into met…
…rics_do_over_main_branch # Conflicts: # rllib/algorithms/algorithm.py # rllib/algorithms/ppo/ppo.py # rllib/env/single_agent_env_runner.py # rllib/utils/metrics/__init__.py # rllib/utils/test_utils.py
Commits on Apr 25, 2024
Commits on Apr 26, 2024
Commits on Apr 27, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_03_learner_on_new_metrics_logger Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/env/multi_agent_env_runner.py # rllib/env/single_agent_env_runner.py
Commits on Apr 28, 2024
-
learns SAC pendulum in 5000ts ~60sec upto better than -300
Signed-off-by: sven1977 <[email protected]>
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_03_learner_on_new_metrics_logger
-
still learns SAC pendulum in 5000ts ~60sec upto better than -300
- also fixed torch tensors showing up in result dicts Signed-off-by: sven1977 <[email protected]>
-
still learns SAC pendulum in 5000ts ~60sec upto better than -300
- also fixed target net update metrics flaws and inheritance chain for DQN vs SAC. Signed-off-by: sven1977 <[email protected]>
-
still learns SAC pendulum in 7000ts ~60sec upto better than -200
- also fixed NaN stats from Envrunner (EnvRunner should never publish NaN-eps-returns for unfinished episodes -> leads to a lot of other problems esepcially with rollout_frag_len=1-style algos, like DQN and SAC). Signed-off-by: sven1977 <[email protected]>
Commits on Apr 29, 2024
-
- after additional_update fixes/cleanups Signed-off-by: sven1977 <[email protected]>
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_03_learner_on_new_metrics_logger Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/algorithms/dqn/dqn.py # rllib/algorithms/dreamerv3/tf/dreamerv3_tf_learner.py # rllib/algorithms/sac/torch/sac_torch_learner.py # rllib/core/learner/learner.py # rllib/core/learner/tests/test_learner_group.py
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_03_learner_on_new_metrics_logger
Commits on Apr 30, 2024
-
- added tensor-mode to MetricsLogger, during which tensors can be logged. - tensor-mode requires to be exited (upon which a tensor dict is returned with the numpy/eager values in the tensors). - these eager/numpy values must then be assigned again into the MetricsLogger, so we can properly reduce. Signed-off-by: sven1977 <[email protected]>
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_03_learner_on_new_metrics_logger Signed-off-by: sven1977 <[email protected]> # Conflicts: # rllib/algorithms/ppo/ppo_learner.py # rllib/algorithms/ppo/tf/ppo_tf_learner.py # rllib/algorithms/ppo/torch/ppo_torch_learner.py
Commits on May 1, 2024
-
- working w/ Atari videos on WandB now (used faster training configs …
…for PPO). Signed-off-by: sven1977 <[email protected]>
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_04_env_rendering_example_script
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_04_env_rendering_example_script Signed-off-by: sven1977 <[email protected]> # Conflicts: # doc/source/rllib/package_ref/learner.rst # rllib/utils/metrics/metrics_logger.py # rllib/utils/metrics/stats.py
Commits on May 2, 2024
-
Merge branch 'master' of https://github.com/ray-project/ray into metr…
…ics_do_over_04_env_rendering_example_script