-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve LLM's tool awareness #1657
Comments
I have solved the name issue with a const with tool name in the original json grammar, i'm not familiar with PR 's in github but will try it in the next days, otherwise i Can leave you the changes there are small and a little few. With that changes the output is 100% aligned with the open ai specs. regards |
in the server.rs the extraction of the name and the tool parser looks like this:
and the input grammar generator looks like:
Added a notify function by default for fallback cases, with that the llm can notify errores, lack of conext information to select a tool, or other kind of errors to avoid selectings wrong tools. Also updated the tool prompt to: Also added a random generator for the call_id number:
its my first time in rust, hope to do it acceptable |
This PR makes tool calling aware of the name of the function selected. Fixes: #1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
This PR makes tool calling aware of the name of the function selected. Fixes: huggingface/text-generation-inference#1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
This PR makes tool calling aware of the name of the function selected. Fixes: huggingface#1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
This PR makes tool calling aware of the name of the function selected. Fixes: huggingface#1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
This PR makes tool calling aware of the name of the function selected. Fixes: huggingface#1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
This PR makes tool calling aware of the name of the function selected. Fixes: huggingface/text-generation-inference#1657 Thank you @puppetm4st3r for the helpful snippets, large parts of this PR are simply refactors of the code shared 🙏 **opening draft PR because small tweaks are needed before merging
Feature request
Currently the tools feature does not return the name of the chosen function. This is due to how tools/functions are implemented in TGI (by constrained generation vs fine tuned model).
It's been proposed that the internal structure is updated to force the generation to include names #1650 and overall the function mechanism may be improved.
Opening this issue as a place for others thoughts/idea/uses on how to improve tools to be as useful as possible.
The text was updated successfully, but these errors were encountered: