Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Move learning_starts logic into execution plans #26032

Merged
merged 55 commits into from
Aug 11, 2022
Merged

[RLlib] Move learning_starts logic into execution plans #26032

merged 55 commits into from
Aug 11, 2022

Commits on Jun 23, 2022

  1. renaming

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    996042e View commit details
    Browse the repository at this point in the history

  2. renaming

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    1a3f972 View commit details
    Browse the repository at this point in the history

  3. lint

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    6ec6d7c View commit details
    Browse the repository at this point in the history

  4. utils fix

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    fb6f2a0 View commit details
    Browse the repository at this point in the history

  5. learning-starts occurences

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    fd022cf View commit details
    Browse the repository at this point in the history

  6. num_ts_added_before_sampling_starts in recsim example

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    9d32f06 View commit details
    Browse the repository at this point in the history

  7. Adds extensive description to all occurences

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    4c75280 View commit details
    Browse the repository at this point in the history

  8. typo

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    ef2396a View commit details
    Browse the repository at this point in the history

  9. Merge branch 'master' into learningstarts

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    e926355 View commit details
    Browse the repository at this point in the history

  10. fix maddpg test config

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 23, 2022
    Configuration menu
    Copy the full SHA
    90cec5c View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2022

  1. merge master again

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jun 24, 2022
    Configuration menu
    Copy the full SHA
    5633595 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. Merge branch 'master' of https://github.com/ray-project/ray into lear… 

    …ningstarts
    @sven1977
    sven1977 committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    2224a0b View commit details
    Browse the repository at this point in the history

  2. wip

    @sven1977
    sven1977 committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    0254a46 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2022

  1. Merge branch 'master' of https://github.com/ray-project/ray into lear… 

    …ningstarts
    @sven1977
    sven1977 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    af3e899 View commit details
    Browse the repository at this point in the history

  2. LINT

    @sven1977
    sven1977 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    a9aed3d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of https://github.com/ray-project/ray into lear… 

    …ningstarts
    @sven1977
    sven1977 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    daf408c View commit details
    Browse the repository at this point in the history

  4. wip

    @sven1977
    sven1977 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    ad22d3a View commit details
    Browse the repository at this point in the history

  5. wip

    @sven1977
    sven1977 committed Jun 30, 2022
    Configuration menu
    Copy the full SHA
    bfc5f08 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2022

  1. merge master

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    7a41dcd View commit details
    Browse the repository at this point in the history

  2. rename to min_size

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    777ffb2 View commit details
    Browse the repository at this point in the history

  3. fix recsim example

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    6b5d9cd View commit details
    Browse the repository at this point in the history

  4. typo

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 9, 2022
    Configuration menu
    Copy the full SHA
    9ff64ed View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2022

  1. restore recsim example

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 10, 2022
    Configuration menu
    Copy the full SHA
    b404c8b View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2022

  1. Merge branch 'master' into learningstarts

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    74f14e2 View commit details
    Browse the repository at this point in the history

  2. merge master

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    d52320f View commit details
    Browse the repository at this point in the history

  3. restores alpha_zero exec plan

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    0f52df3 View commit details
    Browse the repository at this point in the history

  4. remove unneeded params from offline learning examples

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 18, 2022
    Configuration menu
    Copy the full SHA
    6389080 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2022

  1. move arg out of replay buffer config

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    ab5bbbd View commit details
    Browse the repository at this point in the history

  2. move logic into training steps

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jul 24, 2022
    Configuration menu
    Copy the full SHA
    ba822cd View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2022

  1. merge master 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    53daa99 View commit details
    Browse the repository at this point in the history

  2. fix sac and remove replay_starts in buffers 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    16536f9 View commit details
    Browse the repository at this point in the history

  3. wip 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    9218817 View commit details
    Browse the repository at this point in the history

  4. proper training configs 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    e975492 View commit details
    Browse the repository at this point in the history

  5. fixed examples failing due to mismatch wit old_learning_starts parame… 

    …ter; fixed test_store_to_replay_local

Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    b3b5bbb View commit details
    Browse the repository at this point in the history

  6. moved learning starts logic into a sampling while loop 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 1, 2022
    Configuration menu
    Copy the full SHA
    965ddd9 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. fixed compilation tests and simpleq 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    f036fe0 View commit details
    Browse the repository at this point in the history

  2. fix min_size occurences in tuned examples 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    86c0caf View commit details
    Browse the repository at this point in the history

  3. fix apex long_running 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    3049791 View commit details
    Browse the repository at this point in the history

  4. self review 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    2e609d2 View commit details
    Browse the repository at this point in the history

  5. revert 965ddd9 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    107f062 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2022

  1. initial 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    5b0b5aa View commit details
    Browse the repository at this point in the history

  2. merge master 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    acc45ab View commit details
    Browse the repository at this point in the history

  3. Merge branch 'sequencesandprios' into learningstarts

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    f902a94 View commit details
    Browse the repository at this point in the history

  4. fix test_execution, checkpoint_restore and test_custom_resources and … 

    …apex test

Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    19bfa50 View commit details
    Browse the repository at this point in the history

  5. initial 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    694d2cf View commit details
    Browse the repository at this point in the history

  6. Merge branch 'apextestmintrainsteps' into learningstarts

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    09e32d4 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2022

  1. better explanation of what we count by 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    ec738ed View commit details
    Browse the repository at this point in the history

  2. More explicit docstring for config args 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    5116ce1 View commit details
    Browse the repository at this point in the history

  3. Removed unnecessary dreamer line 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    c97ba90 View commit details
    Browse the repository at this point in the history

  4. cleared unneeded logic out of dqn, simpleq and qmix exec plan 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    9cecc17 View commit details
    Browse the repository at this point in the history

  5. --amend 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 9, 2022
    Configuration menu
    Copy the full SHA
    303e593 View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2022

  1. fix ddpg config object in compilation test 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    2be321e View commit details
    Browse the repository at this point in the history

  2. fixed ddpg loss function config 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    6845422 View commit details
    Browse the repository at this point in the history

  3. multi agent buffer arg fixed 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    622d148 View commit details
    Browse the repository at this point in the history

  4. remove replay_buffer_config from td3 sample 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    9902d8e View commit details
    Browse the repository at this point in the history