Add OCI Generative AI tool calling support #16888
Conversation
dosubot
bot
added
the
size:XL
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
| @@ -219,6 +190,7 @@ def _get_all_kwargs(self, **kwargs: Any) -> Dict[str, Any]: | |||
| **kwargs, | |||
| } | |||
|
|
|||
| @deprecated("Deprecated in favor of `chat`, which should be used instead.") | |||
There was a problem hiding this comment.
technically, you can make complete call chat -- we have utils to make this very easy. A few llms do this
There was a problem hiding this comment.
Thank you @logan-markewich Agreed. But basically all the models that support completion have been deprecated in our service. Let me take a look at other vendors.
| async def achat( | ||
| self, messages: Sequence[ChatMessage], **kwargs: Any | ||
| ) -> ChatResponse: | ||
| # do synchronous chat for now | ||
| return self.chat(messages, **kwargs) | ||
| raise NotImplementedError("Async chat is not implemented yet.") |
There was a problem hiding this comment.
There's no async client methods we can implement?
There was a problem hiding this comment.
@logan-markewich Our Python SDK doesn't have async support natively. We probably need to re-evaluation how we gonna implement async. Probably will be another PR as well.
| **chat_params, | ||
| } | ||
|
|
||
| def chat_with_tools( |
There was a problem hiding this comment.
Technically you shouldn't need to implement chat_with_tools or stream_chat_with_tools -- the idea is that chat and stream chat already handle tools being passed in/out, and all you need to implement is _prepare_chat_with_tools() and get_tool_calls_from_response() and maybe _validate_chat_with_tools_response()
Check out anthropic or Ollama for a decent example of this
Description
Adding tool calling support for chat models offered by OCI Generative AI services.
This is an update to our last PR #14080.
Fixes # (issue)
N/A
New Package?
No
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
How Has This Been Tested?
Suggested Checklist:
make format; make lintto appease the lint gods