Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[build_base][RLlib] APPO TF with RLModule and Learner API #33310

Merged
merged 51 commits into from
Mar 26, 2023
Merged

[build_base][RLlib] APPO TF with RLModule and Learner API #33310

merged 51 commits into from
Mar 26, 2023

Commits on Mar 13, 2023

  1. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 13, 2023
    Configuration menu
    Copy the full SHA
    517e0d6 View commit details
    Browse the repository at this point in the history

  2. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 13, 2023
    Configuration menu
    Copy the full SHA
    293ac57 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2023

  1. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    996d6af View commit details
    Browse the repository at this point in the history

  2. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    d83f4de View commit details
    Browse the repository at this point in the history

  3. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    3f4e27c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 14, 2023
    Configuration menu
    Copy the full SHA
    3495576 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2023

  1. Make all TfModels tf.keras.Models 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    606f093 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'fixtensorflowacmodels' into appo_tf

    @avnishn
    avnishn committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    134593e View commit details
    Browse the repository at this point in the history

  3. Running appo 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 15, 2023
    Configuration menu
    Copy the full SHA
    8c5ae9b View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2023

  1. Lint, small updates 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    8a2a0f5 View commit details
    Browse the repository at this point in the history

  2. Move adding params to learner hps to validate in order to be compatib… 

    …le with rllib yaml files

Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    efc74fe View commit details
    Browse the repository at this point in the history

  3. Move adding params to learner hps to validate in order to be compatib… 

    …le with rllib yaml files

Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    13548c1 View commit details
    Browse the repository at this point in the history

  4. Move learner_hp assignment from builder functions to validate 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    cd1270c View commit details
    Browse the repository at this point in the history

  5. Merge branch 'move_learner_hp_assignment' into appo_tf

    @avnishn
    avnishn committed Mar 16, 2023
    Configuration menu
    Copy the full SHA
    1f00e42 View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2023

  1. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 17, 2023
    Configuration menu
    Copy the full SHA
    2ec8d08 View commit details
    Browse the repository at this point in the history

  2. Temp 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 17, 2023
    Configuration menu
    Copy the full SHA
    62753d1 View commit details
    Browse the repository at this point in the history

  3. Clip is ratio 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 17, 2023
    Configuration menu
    Copy the full SHA
    6c91172 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 17, 2023
    Configuration menu
    Copy the full SHA
    61be19b View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2023

  1. Wrote appo tf policy rlm which has working loss but isn't seemingly u… 

    …pdating?

Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 19, 2023
    Configuration menu
    Copy the full SHA
    f0ea920 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 19, 2023
    Configuration menu
    Copy the full SHA
    47d4b5a View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2023

  1. Add option for minibatching in impala/appo with the learner group 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    fb69db2 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 20, 2023
    Configuration menu
    Copy the full SHA
    bb0daeb View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2023

  1. Store most recent result for results reporting 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    68cd9df View commit details
    Browse the repository at this point in the history

  2. dmc wrapper types 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    5008690 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'appo_tf' of https://github.com/avnishn/ray into appo_tf

    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    064adee View commit details
    Browse the repository at this point in the history

  4. ADd back in UpdateTargetAndKL 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    b015015 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'appo_tf' of https://github.com/avnishn/ray; branch 'mas… 

    …ter' of https://github.com/ray-project/ray into appo_tf
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    7842087 View commit details
    Browse the repository at this point in the history

  6. Fix broken tests 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    2b9fec4 View commit details
    Browse the repository at this point in the history

  7. More tf related fixes 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    3fb3615 View commit details
    Browse the repository at this point in the history

  8. More tf related fixes 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    31adb97 View commit details
    Browse the repository at this point in the history

  9. Fix impala test 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    25fdcac View commit details
    Browse the repository at this point in the history

  10. Fixing remaining broken tests 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 21, 2023
    Configuration menu
    Copy the full SHA
    c89e151 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2023

  1. More tf related fixes 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    2f0cea9 View commit details
    Browse the repository at this point in the history

  2. More tf fixes with try catch 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    d61d198 View commit details
    Browse the repository at this point in the history

  3. Addressing comments 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 22, 2023
    Configuration menu
    Copy the full SHA
    e83b0d4 View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2023

  1. Address comments 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    2a5acbe View commit details
    Browse the repository at this point in the history

  2. Ad rl module with target networks mixin interface 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 23, 2023
    Configuration menu
    Copy the full SHA
    44bf47a View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2023

  1. Temp 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    3f113e0 View commit details
    Browse the repository at this point in the history

  2. Address comments 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    5746a8f View commit details
    Browse the repository at this point in the history

  3. Address comments 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    6d28903 View commit details
    Browse the repository at this point in the history

  4. Address comments 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    2b3bfb4 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    288620d View commit details
    Browse the repository at this point in the history

  6. Fix broken import 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    24a1d6e View commit details
    Browse the repository at this point in the history

  7. Lint 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    9594a6c View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    b8e4ec2 View commit details
    Browse the repository at this point in the history

  9. Touching a file 

    Signed-off-by: avnishn <[email protected]>
    @avnishn
    avnishn committed Mar 24, 2023
    Configuration menu
    Copy the full SHA
    3837d36 View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2023

  1. triggering the tests 

    Signed-off-by: Kourosh Hakhamaneshi <[email protected]>
    @kouroshHakha
    kouroshHakha committed Mar 25, 2023
    Configuration menu
    Copy the full SHA
    2d6ac06 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into appo_tf

    @avnishn
    avnishn committed Mar 26, 2023
    Configuration menu
    Copy the full SHA
    5fabfc6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'appo_tf' of https://github.com/avnishn/ray into appo_tf

    @avnishn
    avnishn committed Mar 26, 2023
    Configuration menu
    Copy the full SHA
    117d9dd View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into appo_tf

    @kouroshHakha
    kouroshHakha committed Mar 26, 2023
    Configuration menu
    Copy the full SHA
    68a19ab View commit details
    Browse the repository at this point in the history

  4. Merge branch 'appo_tf' of github.com:avnishn/ray into appo_tf

    @kouroshHakha
    kouroshHakha committed Mar 26, 2023
    Configuration menu
    Copy the full SHA
    9e0d54b View commit details
    Browse the repository at this point in the history