Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix qwen lm_head matrix dimmention alignment #7402

Merged

Conversation

WAI-clear
Copy link
Contributor

PR types

Bug fixes

PR changes

Models

Description

修复了qwen Torch2Paddle 参数自动转换 lm_head权重参数维度不对齐的问题

@CLAassistant
Copy link

CLAassistant commented Nov 8, 2023

CLA assistant check
All committers have signed the CLA.

Copy link
Member

@gongel gongel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

codecov bot commented Nov 8, 2023

Codecov Report

Merging #7402 (e1f9fbd) into develop (c013ccb) will decrease coverage by 0.07%.
Report is 7 commits behind head on develop.
The diff coverage is 36.11%.

@@             Coverage Diff             @@
##           develop    #7402      +/-   ##
===========================================
- Coverage    58.60%   58.54%   -0.07%     
===========================================
  Files          574      574              
  Lines        84866    85048     +182     
===========================================
+ Hits         49739    49791      +52     
- Misses       35127    35257     +130     
Files Coverage Δ
paddlenlp/peft/lora/__init__.py 100.00% <ø> (ø)
paddlenlp/transformers/qwen/modeling.py 81.76% <100.00%> (+0.03%) ⬆️
paddlenlp/transformers/utils.py 68.36% <80.00%> (+1.29%) ⬆️
paddlenlp/utils/serialization.py 89.16% <97.43%> (+0.38%) ⬆️
paddlenlp/transformers/model_utils.py 63.77% <73.33%> (+0.09%) ⬆️
paddlenlp/transformers/llama/modeling.py 67.37% <0.00%> (-0.29%) ⬇️
paddlenlp/peft/lora/lora_model.py 62.50% <15.00%> (-3.45%) ⬇️
paddlenlp/peft/prefix/prefix_model.py 56.52% <15.00%> (-3.57%) ⬇️
paddlenlp/peft/lora/lora_quant_layers.py 38.12% <15.45%> (-49.88%) ⬇️

@gongel gongel merged commit 0771a88 into PaddlePaddle:develop Nov 9, 2023
9 checks passed
Copy link

paddle-bot bot commented Nov 10, 2023

Thanks for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants