Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIR] Add set_preprocessor method to Checkpoint #31721

Merged
merged 3 commits into from
Jan 18, 2023

Conversation

amogkam
Copy link
Contributor

@amogkam amogkam commented Jan 18, 2023

Signed-off-by: amogkam [email protected]

Adds a set_preprocessor method to the Checkpoint class, analogous to the existing get_checkpoint method. This is necessary to easily specify preprocessors to Checkpoints when not using any of the existing framework-specific checkpoints.

Why are these changes needed?

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: amogkam <[email protected]>
@amogkam amogkam changed the title [AIR] Add set_checkpoint to Checkpoint [AIR] Add set_checkpoint method to Checkpoint Jan 18, 2023
python/ray/air/checkpoint.py Show resolved Hide resolved
python/ray/air/checkpoint.py Outdated Show resolved Hide resolved
doc/source/ray-air/predictors.rst Outdated Show resolved Hide resolved
Signed-off-by: amogkam <[email protected]>
Signed-off-by: amogkam <[email protected]>
@Yard1
Copy link
Member

Yard1 commented Jan 18, 2023

Sorry, do you mean set_preprocessor and not set_checkpoint?

@amogkam amogkam changed the title [AIR] Add set_checkpoint method to Checkpoint [AIR] Add set_preprocessor method to Checkpoint Jan 18, 2023
@amogkam
Copy link
Contributor Author

amogkam commented Jan 18, 2023

@Yard1 yes sorry-- udpated the title

Copy link
Contributor

@krfricke krfricke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM


def testObjectRefCheckpointSetPreprocessor(self):
ckpt = Checkpoint.from_dict({"x": 1})
ckpt = ray.get(ray.put(ckpt))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this ckpt is an object ref checkpoint. When I checkedckpt._obj_ref here, I got None.

Is this what we want?

obj_ref = ray.put({"x": 1})
ckpt = Checkpoint.from_object_ref(obj_ref)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from_object_ref is deprecated in favor of ray.get?

Seems like the _obj_ref path is not used anymore at all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, you're right. I'll open a PR to remove the dead code.

@amogkam
Copy link
Contributor Author

amogkam commented Jan 18, 2023

Failing test is also failing on master, going to merge.

@amogkam amogkam merged commit 109a2a2 into ray-project:master Jan 18, 2023
@amogkam amogkam deleted the checkpoint-set-preprocessor branch January 18, 2023 20:57
andreapiso pushed a commit to andreapiso/ray that referenced this pull request Jan 22, 2023
Signed-off-by: amogkam [email protected]

Adds a set_preprocessor method to the Checkpoint class, analogous to the existing get_checkpoint method. This is necessary to easily specify preprocessors to Checkpoints when not using any of the existing framework-specific checkpoints.

Signed-off-by: Andrea Pisoni <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants