Use system message in chat instruct #4648
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist:
This PR allows instruction templates to insert their system message into the context of the prompt in chat-instruct mode.
As an example, before this change, the constructed prompt with the Llama-v2 instruction template in chat-instruct mode would look like this:
Note that verbatim string
<|system-message|>
is provided to the tokenizer instead of the system message itself.With this change, the constructed prompt with Llama-v2 instruction template in chat-instruct mode looks like this:
The system message in the Llama-v2 instruction template is "Answer the questions." and it is now correctly included in the prompt.
These examples use the Llama-v2 instruction template, but the changes apply to any instruction template that has the string
<|system-message|>
in its Context field. If the Custom system message field is not empty, its contents will be used instead of the System message field.