Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding state parser utility that can be used for retrieving and modifying worker states #1278

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

gokulavasan
Copy link
Contributor

Summary: Allows users to get the worker states out of the state dict. It can be used if users want to modify the state offline. Starts off with very preliminary set of methods aimed at supporting usecases where number of workers need to change.

Test Plan: Unit tests

Reviewers:

Subscribers:

Tasks:

Tags:

Please read through our contribution guide prior to
creating your pull request.

  • Note that there is a section on requirements related to adding a new DataPipe.

Fixes #{issue number}

Changes

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 18, 2024
Copy link

pytorch-bot bot commented Jun 18, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/data/1278

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit cb10f71 with merge base 958eeb0 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

logger = logging.getLogger(__name__)


class StateParserUtil:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we keep these as functions? I'm not sure of the benefit of having a class here

Copy link
Contributor

@andrewkho andrewkho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think pure functions are easier to reason about and harder to screw up, let me know if there's other reasons you want to keep them in a class

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants