[INFER][LLM] Support qwen in fined grained dybatch v1

[DanGuge]

PR types

PR changes

Description

• 可以通过以下命令执行动态图推理

python predictor.py \
    --model_name_or_path "qwen/qwen-7b" \
    --inference_model \
    --dtype "float16"

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: 213 lines in your changes are missing coverage. Please review.

Comparison is base (b02c716) 57.48% compared to head (4d478e8) 57.12%.
Report is 56 commits behind head on develop.

Files	Patch %	Lines
...ddlenlp/experimental/transformers/qwen/modeling.py	0.00%	211 Missing ⚠️
paddlenlp/experimental/transformers/__init__.py	0.00%	1 Missing ⚠️
...ddlenlp/experimental/transformers/qwen/__init__.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #7644      +/-   ##
===========================================
- Coverage    57.48%   57.12%   -0.37%     
===========================================
  Files          583      587       +4     
  Lines        87187    88190    +1003     
===========================================
+ Hits         50123    50376     +253     
- Misses       37064    37814     +750     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wj-Mcat]

从 PaddleNLP-CI 的日志中看来是精度没有对齐

[DanGuge]

从 PaddleNLP-CI 的日志中看来是精度没有对齐

• https://github.com/PaddlePaddle/PaddleNLP/blob/develop/tests/llm/test_predictor.py#L71
• 此处的full_match是要求完全一致吗？我自己测试在小数点后几位是有区别的

[DanGuge]

• qwen的非inference_model实现，在多batch推理时精度存在问题
• 运行目录：PaddleNLP/llm

qwen 推理

• batch_size = 1

#!/bin/bash

python3 predictor.py \
    --model_name_or_path qwen/qwen-7b \
    --decode_strategy greedy_search \
    --batch_size 1 \
    --dtype float16

• batch_size = 2

#!/bin/bash

python3 predictor.py \
    --model_name_or_path qwen/qwen-7b \
    --decode_strategy greedy_search \
    --batch_size 2 \
    --dtype float16

qwen inference model 推理

• batch_size = 1

#!/bin/bash

python3 predictor.py \
    --model_name_or_path qwen/qwen-7b \
    --decode_strategy greedy_search \
    --batch_size 1 \
    --inference_model \
    --dtype float16

• batch_size = 2

#!/bin/bash

python3 predictor.py \
    --model_name_or_path qwen/qwen-7b \
    --decode_strategy greedy_search \
    --batch_size 2 \
    --inference_model \
    --dtype float16

cc @wj-Mcat

[wj-Mcat]

LGTM

[wawltor]

LGTM