Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[air] pyarrow.fs persistence (8/n): ray.train.Checkpoint restore: resume_from_checkpoint #38143

Merged
merged 113 commits into from
Aug 9, 2023
Merged

[air] pyarrow.fs persistence (8/n): ray.train.Checkpoint restore: resume_from_checkpoint #38143

merged 113 commits into from
Aug 9, 2023

Commits on Jul 27, 2023

  1. Pipe storage context to Trainable (used now for Trainable syncing) 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    abb1307 View commit details
    Browse the repository at this point in the history

  2. Don't use the storage context in the trial/trainable 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    f6ff90a View commit details
    Browse the repository at this point in the history

  3. Disable all trainable syncing in new codepath 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    562369f View commit details
    Browse the repository at this point in the history

  4. Pipe storage context to Train workers (not actually used yet) 

    Signed-off-by: Justin Yu <[email protected]>

Update to use the new checkpoint id attribute

Signed-off-by: Justin Yu <[email protected]>

Add todo comment to remove legacy path

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    95a3d20 View commit details
    Browse the repository at this point in the history

  5. Fix race condition for setting checkpoint_uri 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    484e67f View commit details
    Browse the repository at this point in the history

  6. Fix cyclical import 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    2148669 View commit details
    Browse the repository at this point in the history

  7. Add simple trainer test 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    8c856b8 View commit details
    Browse the repository at this point in the history

  8. Add legacy prefix to train session checkpoint uri 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    78c525f View commit details
    Browse the repository at this point in the history

  9. Add new checkpoint class 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    e97f471 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. New train session report implementation using new checkpoint 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    64945be View commit details
    Browse the repository at this point in the history

  2. Simplify checkpoint propagation from user code (in worker) -> trainer… 

    … -> driver

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    c6480c9 View commit details
    Browse the repository at this point in the history

  3. New tune session.report 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    c681ccb View commit details
    Browse the repository at this point in the history

  4. Save direction works with new checkpoint API 

    Signed-off-by: Justin Yu <[email protected]>

Fix lint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    795bafe View commit details
    Browse the repository at this point in the history

  5. Update test with e2e trainer test 

    Signed-off-by: Justin Yu <[email protected]>

Fix lint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    8a084bc View commit details
    Browse the repository at this point in the history

  6. Make callback supporting new checkpoint a todo for now 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    725d802 View commit details
    Browse the repository at this point in the history

  7. Remove unnecessary comment 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    877acb9 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/storage_context_to_worker_temp
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    ee4ccbd View commit details
    Browse the repository at this point in the history

  9. Separate out the new set checkpoint id from the old set checkpoint uri 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    88042b3 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    a5eeab2 View commit details
    Browse the repository at this point in the history

  2. Update id -> index 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    a6cd9dc View commit details
    Browse the repository at this point in the history

  3. Address comments on error to raise with old ckpt type 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    01f34bb View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Move checkpoint upload logic to a helper fn of storage ctx 

    Signed-off-by: Justin Yu <[email protected]>

Fix lint for session.py

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    65e7a27 View commit details
    Browse the repository at this point in the history

  2. Drop a checkpoint marker after uploading 

    Signed-off-by: Justin Yu <[email protected]>

Fix lint for storage.py

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    f2a4c36 View commit details
    Browse the repository at this point in the history

  3. Add a simplified checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    49ee126 View commit details
    Browse the repository at this point in the history

  4. Fixes to checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    ffa0dd4 View commit details
    Browse the repository at this point in the history

  5. Add unit test for simplified checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    15553f7 View commit details
    Browse the repository at this point in the history

  6. Full test coverage 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    00cc9d7 View commit details
    Browse the repository at this point in the history

  7. Add a simplified checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    cb5990e View commit details
    Browse the repository at this point in the history

  8. Fixes to checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    2db9aae View commit details
    Browse the repository at this point in the history

  9. Add unit test for simplified checkpoint manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    a2067b7 View commit details
    Browse the repository at this point in the history

  10. Full test coverage 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    f1216f2 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/simplified_ckpt_manager
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    d4243e6 View commit details
    Browse the repository at this point in the history

  12. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    6699d81 View commit details
    Browse the repository at this point in the history

  13. Simplify even more 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    9b9ff34 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/simplified_ckpt_manager
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    83aecd9 View commit details
    Browse the repository at this point in the history

  15. Patch fix for circular imports 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    913af10 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Use new checkpoint manager in Tune ckpt book-keeping 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    6b5d34e View commit details
    Browse the repository at this point in the history

  2. Update result to return a train.Checkpoint to the user 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    24f441a View commit details
    Browse the repository at this point in the history

  3. Update e2e test to try multiple ckpt configs for trainer test 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    504ed54 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    1992161 View commit details
    Browse the repository at this point in the history

  5. Fix lint for trial.py 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    b9eb88f View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/simplified_ckpt_manager
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    a6115b3 View commit details
    Browse the repository at this point in the history

  7. Rename _TrackedCheckpoint -> _TrainingResult 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    7cc74d9 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/simplified_ckpt_manager
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    6a0e1fb View commit details
    Browse the repository at this point in the history

  9. Merge branch 'air/persistence/simplified_ckpt_manager' into air/persi… 

    …stence/new_checkpoint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    4662789 View commit details
    Browse the repository at this point in the history

  10. Fixes after merging latest ckpt manager changes 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    8da0477 View commit details
    Browse the repository at this point in the history

  11. Remove prints / convert to logger.debug 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    255b149 View commit details
    Browse the repository at this point in the history

  12. Don't set training iteration as the default checkpoint_score_attr 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    0971aca View commit details
    Browse the repository at this point in the history

  13. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    6e7a873 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    6f6a341 View commit details
    Browse the repository at this point in the history

  15. Fix test to reflect working dir change 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    d9804b0 View commit details
    Browse the repository at this point in the history

  16. Don't upload a .is_checkpoint marker 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    318158f View commit details
    Browse the repository at this point in the history

  17. Add back cwd check 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    a54664c View commit details
    Browse the repository at this point in the history

  18. Update the dir trees + better naming for ckpt shards and artifacts 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    c4263ec View commit details
    Browse the repository at this point in the history

  19. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    0cd7e47 View commit details
    Browse the repository at this point in the history

  20. A different fix for the circular dep 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    3a6eba6 View commit details
    Browse the repository at this point in the history

  21. Update checkpoint -> _checkpoint imports 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    b65e9fe View commit details
    Browse the repository at this point in the history

  22. fix lint 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    b89bd1c View commit details
    Browse the repository at this point in the history

  23. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    cada06a View commit details
    Browse the repository at this point in the history

  24. Revert all changes to ckpt manager 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    3b784d7 View commit details
    Browse the repository at this point in the history

  25. Don't set checkpoint user metadata 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    49c1ead View commit details
    Browse the repository at this point in the history

  26. Remove remaining print 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    7177940 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. Add trial_path property to storage ctx 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    ae8a9ec View commit details
    Browse the repository at this point in the history

  2. Use storage context for all experiment/trial path properties 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    c1c8441 View commit details
    Browse the repository at this point in the history

  3. Don't skip trainer test cases for custom_fs 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5d2ca07 View commit details
    Browse the repository at this point in the history

  4. Split some utilities into helper methods + test for ResultGrid paths 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    1fcfb3f View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/fix_custom_fs_path_expansion
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5e2a933 View commit details
    Browse the repository at this point in the history

  6. Prepend legacy to old path attributes in trial 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    61fdadf View commit details
    Browse the repository at this point in the history

  7. Remove todo 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    d38cd87 View commit details
    Browse the repository at this point in the history

  8. Bump the test size 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    3ba944a View commit details
    Browse the repository at this point in the history

  9. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/new_checkpoint

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    5f71608 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'air/persistence/new_checkpoint' into air/persistence/fi… 

    …x_custom_fs_path_expansion

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    52d6c14 View commit details
    Browse the repository at this point in the history

  11. Clean up experiment path handling 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    9c16120 View commit details
    Browse the repository at this point in the history

  12. Fix for base trainer 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    76468d9 View commit details
    Browse the repository at this point in the history

  13. Fix for base trainer pt 2 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    b17c17e View commit details
    Browse the repository at this point in the history

  14. Add in missing legacy property 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    30e3328 View commit details
    Browse the repository at this point in the history

  15. Prepend legacy to old path attributes in experiment 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    f25ad39 View commit details
    Browse the repository at this point in the history

  16. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/fix_custom_fs_path_expansion
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    e1846ec View commit details
    Browse the repository at this point in the history

  17. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/fix_custom_fs_path_expansion

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    d11ede8 View commit details
    Browse the repository at this point in the history

  18. too much space 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    c99d30c View commit details
    Browse the repository at this point in the history

  19. remove unused var 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    9d11d2d View commit details
    Browse the repository at this point in the history

  20. Fix lint 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    950b991 View commit details
    Browse the repository at this point in the history

Commits on Aug 4, 2023

  1. restore mostly works 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    ed86255 View commit details
    Browse the repository at this point in the history

  2. hacky way of getting checkpoint folders to increment correctly 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    de4b924 View commit details
    Browse the repository at this point in the history

  3. Fix for xgboost trainer 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    e060476 View commit details
    Browse the repository at this point in the history

  4. Fix race as_directory / download file lock race condition 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    bd5c846 View commit details
    Browse the repository at this point in the history

  5. Update test with auto-recovery fault tolerance 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    e51fb17 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/fix_custom_fs_path_expansion
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    eaa26c5 View commit details
    Browse the repository at this point in the history

  7. compute storage_prefix 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    0c3c5c8 View commit details
    Browse the repository at this point in the history

  8. Remove '_path' properties from storage 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    217af77 View commit details
    Browse the repository at this point in the history

  9. Move exp dir name helper to storage ctx 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    8e2330c View commit details
    Browse the repository at this point in the history

  10. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/fix_custom_fs_path_expansion
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    7502cca View commit details
    Browse the repository at this point in the history

  11. Fix bugs causing broken CI 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    f3f22fd View commit details
    Browse the repository at this point in the history

  12. Fix renamed attribute in mock test class 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    c6c3dfe View commit details
    Browse the repository at this point in the history

  13. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/restore_new_checkpoint_autoft
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    4e56bd0 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'air/persistence/fix_custom_fs_path_expansion' into air/… 

    …persistence/restore_new_checkpoint_autoft

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 4, 2023
    Configuration menu
    Copy the full SHA
    314e8bd View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2023

  1. fix storage attr setting to only happen if ff enabled 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    36464af View commit details
    Browse the repository at this point in the history

  2. cleanup on errors in as_directory 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    6e73f6e View commit details
    Browse the repository at this point in the history

  3. Support + test resume_from_checkpoint 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    c7de72a View commit details
    Browse the repository at this point in the history

  4. Fix result grid bug when no checkpoints saved 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    db016da View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/restore_new_checkpoint_autoft

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    bcbcec9 View commit details
    Browse the repository at this point in the history

  6. fix merge conflict remainder 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 5, 2023
    Configuration menu
    Copy the full SHA
    d7497e1 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Recover trainable metadata from last_result rather than .tune_metadata 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    aeb89ba View commit details
    Browse the repository at this point in the history

  2. Fix restore info log 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9556371 View commit details
    Browse the repository at this point in the history

  3. Keep current checkpoint index synchronized on the driver 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    e897eaa View commit details
    Browse the repository at this point in the history

  4. Remove checkpoint dirname parsing 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    3eef417 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/restore_new_checkpoint_autoft
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    0e52384 View commit details
    Browse the repository at this point in the history

  6. Update todo comment 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    89631ab View commit details
    Browse the repository at this point in the history

  7. Fix lint 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    d4e20f2 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'air/persistence/restore_new_checkpoint_autoft' into air… 

    …/persistence/restore_new_checkpoint_rfc
    @justinvyu
    justinvyu committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    877c4ae View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Merge branch 'master' of https://github.com/ray-project/ray into air/… 

    …persistence/restore_new_checkpoint_rfc

Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    d367e8d View commit details
    Browse the repository at this point in the history

  2. Fix e2e test for storage_path=None case 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    d329e1b View commit details
    Browse the repository at this point in the history

  3. Remove unused code 

    Signed-off-by: Justin Yu <[email protected]>
    @justinvyu
    justinvyu committed Aug 8, 2023
    Configuration menu
    Copy the full SHA
    e382e29 View commit details
    Browse the repository at this point in the history