Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib; Offline RL] Offline performance cleanup. #47731

Merged
merged 16 commits into from
Sep 25, 2024
Merged

[RLlib; Offline RL] Offline performance cleanup. #47731

merged 16 commits into from
Sep 25, 2024
+302 −117

Commits on Sep 10, 2024

  1. Added reinitialization of the 'batch_iterator' to 'OfflineData.sample… 

    …'. This was initialized at each iteration and slowed down our 'OfflineData' sampling. Ina ddition tuned all Offline examples for the changes made.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    eb61c8f View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2024

  1. Added logic to materialize the data after mapping batches. Furthermor… 

    …e added an option for users to materialize the dataset if needed and enough memory is available.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 17, 2024
    Configuration menu
    Copy the full SHA
    a48be61 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2024

  1. Intermediate results. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    62d8819 View commit details
    Browse the repository at this point in the history

  2. Intermediate results. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    531b526 View commit details
    Browse the repository at this point in the history

  3. Merged master. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    0f36c96 View commit details
    Browse the repository at this point in the history

  4. Full cleanup of 'OfflineData' and 'OfflineDataPreLearner' using mater… 

    …ialization and reinitialization of iterators in single-learner mode. Furthermore, changed after-mapping batch size to 1 b/c rows are then 'MultiAgentBatches' of 'train_batch_size_per_learner' environment steps each. In addition added two further options to 'AlgorithmConfig' such that users can control memory usage and performance.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    61e44a1 View commit details
    Browse the repository at this point in the history

  5. Fixed an error in MARWIL's loss function that ignored basically the v… 

    …alue function output from 'forward_train' and did therefore not train the value function.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 18, 2024
    Configuration menu
    Copy the full SHA
    b83dd0b View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2024

  1. Fixed a test that was failing. When an iterator should be returned an… 

    …d and is already converted to a generator we need to rebuild it.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    4eae101 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into offline-performance-cleanup

    @simonsays1980
    simonsays1980 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    e3a0224 View commit details
    Browse the repository at this point in the history

  3. Update rllib/algorithms/algorithm_config.py 

    Co-authored-by: Sven Mika <[email protected]>
Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980 @sven1977
    simonsays1980 and sven1977 authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    37109e0 View commit details
    Browse the repository at this point in the history

  4. Update rllib/algorithms/algorithm_config.py 

    Co-authored-by: Sven Mika <[email protected]>
Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980 @sven1977
    simonsays1980 and sven1977 authored Sep 19, 2024
    Configuration menu
    Copy the full SHA
    e26b53e View commit details
    Browse the repository at this point in the history

  5. Added suggestions from @sven1977's review. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    398113b View commit details
    Browse the repository at this point in the history

  6. Fixed a bug with the filesystem in 'OffineData. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    6ddd15d View commit details
    Browse the repository at this point in the history

  7. Set offline data test to 'medium' as it timed out constantly. ALso ad… 

    …ded anonymous filesystem to RLUnplugged example.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    915cce9 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Optimized number of blocks and increased number of reading actors. 

    Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    18cda9f View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. Fixed some bugs in the MARWIL tests due to some changes in the calcul… 

    …ation of advantages.

Signed-off-by: simonsays1980 <[email protected]>
    @simonsays1980
    simonsays1980 committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    2e727f2 View commit details
    Browse the repository at this point in the history