Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[UIE Issue]UIE关系抽取后执行evaluate.py文件发现,对性能评估的问题:关系的类居然是一条一条class 输出的,不应该是都属于一类输出吗 #3224

Closed
dingidng opened this issue Sep 8, 2022 · 5 comments
Assignees
Labels
question Further information is requested stale

Comments

@dingidng
Copy link

dingidng commented Sep 8, 2022

记得很早之前看过一个关于UIE关系抽取后执行evaluate.py文件发现,对性能评估的问题。

以这篇项目为例:
Paddlenlp之UIE关系抽取模型【高管关系抽取为例】
https://aistudio.baidu.com/aistudio/projectdetail/4371345?contributionType=1
项目中结果验证时:
Class Name: 人名
[2022-07-25 00:17:26,608] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
[2022-07-25 00:17:26,641] [ INFO] - -----------------------------
[2022-07-25 00:17:26,641] [ INFO] - Class Name: 公司
[2022-07-25 00:17:26,641] [ INFO] - Evaluation Precision: 1.00000 | Recall: 0.66667 | F1: 0.80000
[2022-07-25 00:17:26,667] [ INFO] - -----------------------------
[2022-07-25 00:17:26,667] [ INFO] - Class Name: 网易公司的高管
[2022-07-25 00:17:26,667] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000
[2022-07-25 00:17:26,693] [ INFO] - -----------------------------
[2022-07-25 00:17:26,693] [ INFO] - Class Name: 百度公司的高管
[2022-07-25 00:17:26,693] [ INFO] - Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000

关系的类居然是一条一条class 输出的,不应该是都属于一类输出吗
百度公司的高管 网易公司的高管 应该输出为:Class Name:高管 Evaluation Precision: 1.00000 | Recall: 1.00000 | F1: 1.00000

不知道更新优化了吗

@dingidng dingidng added the question Further information is requested label Sep 8, 2022
@MachineSheep
Copy link

evaluate.py debug输出的各类别集合和数据集内"prompt"字段内容集合是一样的

@dingidng
Copy link
Author

dingidng commented Sep 8, 2022

evaluate.py debug输出的各类别集合和数据集内"prompt"字段内容集合是一样的

那样不是很不友好呀,关系抽取得prompt好像是 “” xxx的xxx“”
做关系抽取关系数据,希望看到所有“”高管“”关系抽取的ACC等指标,这样一条一条看有点麻烦呀:

@linjieccc linjieccc self-assigned this Sep 8, 2022
@linjieccc
Copy link
Contributor

感谢反馈~我们优化下这部分体验 @dingidng

@github-actions
Copy link

github-actions bot commented Dec 7, 2022

This issue is stale because it has been open for 60 days with no activity. 当前issue 60天内无活动,被标记为stale。

@github-actions github-actions bot added the stale label Dec 7, 2022
@github-actions
Copy link

This issue was closed because it has been inactive for 14 days since being marked as stale. 当前issue 被标记为stale已有14天,即将关闭。

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Dec 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested stale
Projects
None yet
Development

No branches or pull requests

3 participants