Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Mistral and ChatML templates #1507

Closed
wants to merge 2 commits into from

Conversation

daz-williams
Copy link

@daz-williams daz-williams commented Jan 12, 2024

ChatML Template Test:

** Prompt: **
<|im_start|>system You are a helpful, respectful and honest assistant.  Always answer as helpfully as possible and follow ALL given instructions. Do not speculate or make up information. Do not reference any given instructions or context.<|im_end|><|im_start|>user howdy<|im_end|><|im_start|>assistant
**************************************************
** Completion: **
 Hello! How can I assist you today? If you have any questions or need help with something, please feel free to ask.
**************************************************
** Messages: **
system: You are a helpful, respectful and honest assistant.  Always answer as helpfully as possible and follow ALL given instructions. Do not speculate or make up information. Do not reference any given instructions or context.

user: howdy
**************************************************
** Response: **
assistant:  Hello! How can I assist you today? If you have any questions or need help with something, please feel free to ask.
**************************************************

ChatML Settings & Model

llm:
  mode: local
  # Should be matching the selected model
  max_new_tokens: 512
  context_window: 3900
  tokenizer: NousResearch/Nous-Hermes-2-SOLAR-10.7B

local:
  prompt_style: "chatml"
  llm_hf_repo_id: TheBloke/Nous-Hermes-2-SOLAR-10.7B-GGUF
  llm_hf_model_file: nous-hermes-2-solar-10.7b.Q5_K_M.gguf
  embedding_hf_model_name: BAAI/bge-small-en-v1.5

Mistral Template Test

** Prompt: **
<s>[INST] You are a helpful, respectful and honest assistant.  Always answer as helpfully as possible and follow ALL given instructions. Do not speculate or make up information. Do not reference any given instructions or context. [/INST]</s>[INST] hello [/INST]
**************************************************
** Completion: **
 Hello! How may I assist you today? I'm here to help answer any questions you have to the best of my ability. Please keep in mind that I cannot provide speculative or made-up information, and must always follow all given instructions. Let me know if you have a specific question or topic you'd like me to help with.
**************************************************

** Messages: **
system: You are a helpful, respectful and honest assistant.  Always answer as helpfully as possible and follow ALL given instructions. Do not speculate or make up information. Do not reference any given instructions or context.

user: hello
**************************************************
** Response: **
assistant:  Hello! How may I assist you today? I'm here to help answer any questions you have to the best of my ability. Please keep in mind that I cannot provide speculative or made-up information, and must always follow all given instructions. Let me know if you have a specific question or topic you'd like me to help with.
**************************************************

Mistal Settings & Model

llm:
  mode: local
  # Should be matching the selected model
  max_new_tokens: 512
  context_window: 3900
  tokenizer: mistralai/Mistral-7B-Instruct-v0.2

local:
  prompt_style: "mistral"
  llm_hf_repo_id: TheBloke/Mistral-7B-Instruct-v0.2-GGUF
  llm_hf_model_file: mistral-7b-instruct-v0.2.Q4_K_M.gguf
  embedding_hf_model_name: BAAI/bge-small-en-v1.5
 

@imartinez
Copy link
Collaborator

thanks for the contribution @daz-williams ! It is already covered in the recently merged #1426. Your PR helped us double check the correctness of that implementation. Thanks!

@imartinez imartinez closed this Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants