Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add qkv_proj to the saved name list #965

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Add qkv_proj to the saved name list #965

wants to merge 1 commit into from

Conversation

zhenying-liu
Copy link

Update QKV offloading and rename remat policy.

  • Add qkv_proj to offloadable names due to query/key/value fusion
  • Rename remat_policy from qkv_proj_offloaded to save_qkv_proj_offloaded

@zhenying-liu
Copy link
Author

For those models like "default" model don't have qkv_proj, it won't be recognized and there will be an error regarding it. So this CL needs to be further polished.

@zhenying-liu zhenying-liu marked this pull request as draft October 16, 2024 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gobbleturk gobbleturk Awaiting requested review from gobbleturk gobbleturk is a code owner

@jonb377 jonb377 Awaiting requested review from jonb377 jonb377 is a code owner

@khatwanimohit khatwanimohit Awaiting requested review from khatwanimohit khatwanimohit is a code owner

@bvandermoon bvandermoon Awaiting requested review from bvandermoon bvandermoon is a code owner

@vipannalla vipannalla Awaiting requested review from vipannalla vipannalla is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zhenying-liu