Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Add a test for loading checkpoint v1.0 Policies using the new Policy.from_checkpoint(). #29370

Merged
merged 4 commits into from
Oct 25, 2022
Merged

[RLlib] Add a test for loading checkpoint v1.0 Policies using the new Policy.from_checkpoint(). #29370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9038487
Add a test for loading v1 policies using the new Policy.from_checkpoi…
Oct 14, 2022
6a8d407
fix
Oct 14, 2022
038f200
fix
Oct 20, 2022
d44ab0c
fix
Oct 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf/.is_checkpoint
View file
Open in desktop
Empty file.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf/.tune_metadata
View file
Open in desktop
Binary file not shown.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf/algorithm_state.pkl
View file
Open in desktop
Binary file not shown.
Binary file added ...ckward_compat/checkpoints/v1.0/dqn_frozenlake_tf/policies/default_policy/policy_state.pkl
View file
Open in desktop
Binary file not shown.
1 change: 1 addition & 0 deletions ...d_compat/checkpoints/v1.0/dqn_frozenlake_tf/policies/default_policy/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Policy", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
1 change: 1 addition & 0 deletions rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Algorithm", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
Empty file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf2/.is_checkpoint
View file
Open in desktop
Empty file.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf2/.tune_metadata
View file
Open in desktop
Binary file not shown.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf2/algorithm_state.pkl
View file
Open in desktop
Binary file not shown.
Binary file added ...kward_compat/checkpoints/v1.0/dqn_frozenlake_tf2/policies/default_policy/policy_state.pkl
View file
Open in desktop
Binary file not shown.
1 change: 1 addition & 0 deletions ..._compat/checkpoints/v1.0/dqn_frozenlake_tf2/policies/default_policy/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Policy", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
1 change: 1 addition & 0 deletions rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_tf2/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Algorithm", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
Empty file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_torch/.is_checkpoint
View file
Open in desktop
Empty file.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_torch/.tune_metadata
View file
Open in desktop
Binary file not shown.
Binary file added rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_torch/algorithm_state.pkl
View file
Open in desktop
Binary file not shown.
Binary file added ...ard_compat/checkpoints/v1.0/dqn_frozenlake_torch/policies/default_policy/policy_state.pkl
View file
Open in desktop
Binary file not shown.
1 change: 1 addition & 0 deletions ...ompat/checkpoints/v1.0/dqn_frozenlake_torch/policies/default_policy/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Policy", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
1 change: 1 addition & 0 deletions rllib/tests/backward_compat/checkpoints/v1.0/dqn_frozenlake_torch/rllib_checkpoint.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"type": "Algorithm", "checkpoint_version": "1.0", "ray_version": "3.0.0.dev0", "ray_commit": "5ae1638c8797518dc2177d7cf5d327cd9c32d706"}
32 changes: 0 additions & 32 deletions rllib/tests/backward_compat/old_ppo.py
View file
Open in desktop

This file was deleted.

236 changes: 0 additions & 236 deletions rllib/tests/backward_compat/old_registry.py
View file
Open in desktop

This file was deleted.

30 changes: 29 additions & 1 deletion rllib/tests/backward_compat/test_backward_compat.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,39 @@ def test_old_checkpoint_formats(self):

# Also test restoring a Policy from an algo checkpoint.
policies = Policy.from_checkpoint(path_to_checkpoint)
assert "default_policy" in policies
self.assertTrue("default_policy" in policies)

print(algo.train())
algo.stop()

def test_v1_policy_from_checkpoint(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for my understanding: Any reason, why we shouldn't just do this test case in a loop?

B/c you just want to test Policy (not algo)? Python/pickle versions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean merge this test with the one above?
the thing is that I need to test DQN policies here. because that is the one we didn't finish migrating to PolicyV2.
it feels a special case compared to the other backward compatibility tests.
we can also remove this test once I move DQN, etc to PolicyV2.

"""Tests, whether we can load Policy checkpoints for different frameworks."""

# We wouldn't need this test once we get rid of V1 policy implementations.

rllib_dir = Path(__file__).parent.parent.parent
print(f"rllib dir={rllib_dir} exists={os.path.isdir(rllib_dir)}")

for fw in framework_iterator(with_eager_tracing=True):
path_to_checkpoint = os.path.join(
rllib_dir,
"tests",
"backward_compat",
"checkpoints",
"v1.0",
"dqn_frozenlake_" + fw,
"policies",
"default_policy",
)

print(
f"path_to_checkpoint={path_to_checkpoint} "
f"exists={os.path.isdir(path_to_checkpoint)}"
)

policy = Policy.from_checkpoint(path_to_checkpoint)
self.assertTrue(isinstance(policy, Policy))


if __name__ == "__main__":
import pytest
Expand Down