Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs] Mention FSDP in HuggingFaceTrainer docs #28217

Merged
merged 3 commits into from
Sep 7, 2022
Merged

[Docs] Mention FSDP in HuggingFaceTrainer docs #28217

merged 3 commits into from
Sep 7, 2022

Conversation

Yard1
Copy link
Member

@Yard1 Yard1 commented Aug 31, 2022

Signed-off-by: Antoni Baum [email protected]

Why are these changes needed?

Mentions that it is possible to run FSDP using HuggingFaceTrainer.

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@Yard1 Yard1 added this to the Ray AIR milestone Aug 31, 2022
@Yard1 Yard1 requested a review from amogkam August 31, 2022 19:45
amogkam
amogkam approved these changes Aug 31, 2022
View reviewed changes
python/ray/train/huggingface/huggingface_trainer.py Outdated
installed, you can also run FSDP training by specifying the ``fsdp`` argument
in ``TrainingArguments``. For more information on configuring FSDP,
refer to `Hugging Face documentation <https://huggingface.co/docs/transformers/\
main/en/main_classes/trainer#transformers.TrainingArguments.fsdp>`__.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link is broken for me?

@Yard1
Copy link
Member Author

Yard1 commented Aug 31, 2022

@amogkam I think it's due to the line break (\). I just checked, it works fine without it, make sure you remove it when pasting. It also works fine when you click on it in the docs.
https://huggingface.co/docs/transformers/main/en/main_classes/trainer#transformers.TrainingArguments.fsdp

@amogkam
Copy link
Contributor

amogkam commented Aug 31, 2022 •
edited
Loading

Oh of course 🤦

Just a nit, but would it make sense to put the full URL on one line with a noqa? I can imagine other users trying to copy-paste directly without realizing there is a line break.

@Yard1
Copy link
Member Author

Yard1 commented Aug 31, 2022

The noqa would have to apply to the entire docstring, as we can't have a comment inside of it.

@amogkam amogkam merged commit a66a5a1 into ray-project:master Sep 7, 2022
@Yard1 Yard1 deleted the hf_fsdp_docs branch September 7, 2022 10:04
ilee300a pushed a commit to ilee300a/ray that referenced this pull request Sep 12, 2022
@Yard1 @amogkam @ilee300a
[Docs] Mention FSDP in HuggingFaceTrainer docs (ray-project#28217)
543b311 
Mentions that it is possible to run FSDP using HuggingFaceTrainer.

Signed-off-by: Antoni Baum <[email protected]>
Co-authored-by: Amog Kamsetty <[email protected]>
Signed-off-by: ilee300a <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amogkam amogkam amogkam approved these changes

Assignees

@amogkam amogkam

@Yard1 Yard1

Labels
None yet
Projects
None yet
Milestone
Ray AIR
Development

Successfully merging this pull request may close these issues.
2 participants
@Yard1 @amogkam