Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor NormalizedConfigs for GQA #1539

Merged
merged 1 commit into from
Nov 15, 2023
Merged

Refactor NormalizedConfigs for GQA #1539

merged 1 commit into from
Nov 15, 2023

Conversation

michaelbenayoun
Copy link
Member

What does this PR do?

It creates a NormalizedTextConfigWithGQA class that inherints NormalizedTextConfig to handle models that use GQA instead of vanilla MHA.

@michaelbenayoun michaelbenayoun requested review from echarlaix and fxmarty and removed request for echarlaix November 15, 2023 08:54
@fxmarty
Copy link
Contributor

fxmarty commented Nov 15, 2023

was there a bug somewhere?

@michaelbenayoun
Copy link
Member Author

No, not really. But the changes for Mistral are also useful to Llama because it can use GQA as well.

@echarlaix echarlaix merged commit ea4349d into huggingface:main Nov 15, 2023
50 of 52 checks passed
@michaelbenayoun michaelbenayoun deleted the mistral_normalized_config branch November 15, 2023 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@echarlaix echarlaix echarlaix approved these changes

@fxmarty fxmarty Awaiting requested review from fxmarty

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@michaelbenayoun @fxmarty @echarlaix