We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
无
在数据量很大的情况下,pre-tokenize能够节省训练时候花费的时间吧
No response
The text was updated successfully, but these errors were encountered:
https://github.com/hiyouga/LLaMA-Factory/tree/main/examples#preprocess-dataset
Sorry, something went wrong.
No branches or pull requests
Reminder
System Info
无
Reproduction
无
Expected behavior
在数据量很大的情况下,pre-tokenize能够节省训练时候花费的时间吧
Others
No response
The text was updated successfully, but these errors were encountered: