Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Properly serialize and restore StateBufferConnector states for policy stashing #31372

Merged
merged 15 commits into from
Jan 5, 2023
Merged

[RLlib] Properly serialize and restore StateBufferConnector states for policy stashing #31372

merged 15 commits into from
Jan 5, 2023

Commits on Jan 4, 2023

  1. initial 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    24a2414 View commit details
    Browse the repository at this point in the history

  2. initial 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    ed930cb View commit details
    Browse the repository at this point in the history

  3. disable connectors 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    3abe2e9 View commit details
    Browse the repository at this point in the history

  4. Fix EnvRunnerV2's handling of soft_horizon episodes. 

    Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    eae9710 View commit details
    Browse the repository at this point in the history

  5. Enable connectors to see broken tests 

    Signed-off-by: Avnish <[email protected]>
    @avnishn
    avnishn authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    2dce34e View commit details
    Browse the repository at this point in the history

  6. disable connectors again 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    6395b4c View commit details
    Browse the repository at this point in the history

  7. initial 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    857538f View commit details
    Browse the repository at this point in the history

  8. only flip switch 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    daa55a6 View commit details
    Browse the repository at this point in the history

  9. flip 

    Signed-off-by: Artur Niederfahrenhorst <[email protected]>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst authored and Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    063a462 View commit details
    Browse the repository at this point in the history

  10. Serialize StateBufferConnector states in case a policy is stashed and… 

    … restored during the rollout of an single episode.

Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    fd4ad78 View commit details
    Browse the repository at this point in the history

  11. revert some debuggin things. 

    reduce 100 policy size since we are doing a lot of excessive policy resotring at this point.

Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    b4b5b11 View commit details
    Browse the repository at this point in the history

  12. check() should handle byte strings 

    Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    be92e9a View commit details
    Browse the repository at this point in the history

  13. make sure states are cleared when in eval mode. also _PolicyCollector… 

    …Group does not need to initiate a collector for every single policies up-front.

Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    69c67d9 View commit details
    Browse the repository at this point in the history

  14. fix test_algorithm 

    Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 4, 2023
    Configuration menu
    Copy the full SHA
    40adca3 View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2023

  1. wip 

    Signed-off-by: Jun Gong <[email protected]>
    Jun Gong committed Jan 5, 2023
    Configuration menu
    Copy the full SHA
    e44b21e View commit details
    Browse the repository at this point in the history