Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Train] Update documentation to show how to use torchmetrics for metric aggregation #31434

Closed
Yard1 opened this issue Jan 4, 2023 · 0 comments · Fixed by #31498
Closed

[Train] Update documentation to show how to use torchmetrics for metric aggregation #31434

Yard1 opened this issue Jan 4, 2023 · 0 comments · Fixed by #31498
Assignees
@Yard1
Labels
docs An issue or change related to documentation enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks ray-team-created Ray Team created train Ray Train Related Issue

Comments

@Yard1
Copy link
Member

Yard1 commented Jan 4, 2023 •
edited by richardliaw
Loading

Description

For some applications of Ray Train, it may be desirable to report metrics from all workers and/or report aggregations, such as mean and std. While Train currently reports metrics only from rank 0 worker, Torch users can use torchmetrics to aggregate metrics from other workers and report them on rank 0. We should update the documentation to show how that can be done.

Use case

No response
@Yard1 Yard1 added enhancement Request for new feature and/or capability P2 Important issue, but not time-critical train Ray Train Related Issue ray-team-created Ray Team created labels Jan 4, 2023
@Yard1 Yard1 mentioned this issue Jan 4, 2023
Closed
@richardliaw richardliaw changed the title [Train] Update documentation to show how to use torchmetrics for metric aggregation [Train] Update documentation to show how to use `torchmetrics` for metric aggregation Jan 4, 2023
@richardliaw richardliaw added air docs An issue or change related to documentation P1 Issue that should be fixed within a few weeks and removed P2 Important issue, but not time-critical labels Jan 4, 2023 — with Slack
@Yard1 Yard1 changed the title [Train] Update documentation to show how to use `torchmetrics` for metric aggregation [Train] Update documentation to show how to use torchmetrics for metric aggregation Jan 4, 2023
@Yard1 Yard1 self-assigned this Jan 5, 2023
@Yard1 Yard1 mentioned this issue Jan 6, 2023
Merged
7 tasks
@matthewdeng matthewdeng mentioned this issue Mar 16, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Yard1 Yard1

Labels
docs An issue or change related to documentation enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks ray-team-created Ray Team created train Ray Train Related Issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Doc] Improve metrics aggregation Train user guide Yard1/ray
2 participants
@richardliaw @Yard1