We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
llamafactory
我注意到sharegpt格式的数据在训练计算loss时,奇数位当作input不做训练(mask),偶数位认为是llm要生成的结果,需要逐token算loss,还有几个问题想详细了解下。 输入输出是分别把奇偶数位的内容concat吗,因而只能看见(输入)所有位置的内容,训练(输出)所有位置的内容,而不能选择只训练最后一个偶数位内容。
No response
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Reminder
System Info
llamafactory
version: 0.9.0Reproduction
我注意到sharegpt格式的数据在训练计算loss时,奇数位当作input不做训练(mask),偶数位认为是llm要生成的结果,需要逐token算loss,还有几个问题想详细了解下。
输入输出是分别把奇偶数位的内容concat吗,因而只能看见(输入)所有位置的内容,训练(输出)所有位置的内容,而不能选择只训练最后一个偶数位内容。
Expected behavior
No response
Others
No response
The text was updated successfully, but these errors were encountered: