Empty args/inputs when turning streaming on and setting tool choice to any #658
Comments
|
@kwnath thanks for the report and the detailed example notebook. Can you share the python version you're using and the dependency versions you're using? e.g. with I can't reproduce the issue you're seeing, with this script from anthropic import Anthropic
client = Anthropic()
stream = client.messages.create(
model="claude-3-5-sonnet-20240620",
max_tokens=2048,
temperature=0.0,
tools=[
{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
}
},
"required": ["location"],
},
}
],
tool_choice={"type": "any"},
stream=True,
messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
)
for event in stream:
print(event)I get back multiple events |
|
Ah completely forgot to add this! Apologies! |
|
Thanks! I still can't reproduce with the dependencies you listed, I'm getting output like this: Have you tested using the SDK outside of a notebook? I wonder if that's causing issues? |
|
Hmm interesting this is only happening on the notebook. I ran this in a python shell and I get multiple chunks. |
|
Interesting, thanks. Could you share more details on how you have your notebook setup? |
|
Thanks @RobertCraigie I'll have a deeper look into how we're setting up our notebook but for now I'm happy to close this issue and update when I do find something in our setup! |
|
Thanks for confirming! Did you manage to determine what was causing issues in your setup? |
|
@kwnath did you manage to figure anything out? |
|
Hey Robert, I haven't yet, I had run a few tests around this and am still experiencing a few function calls without args. Will be investigating a bit more deeply today. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
When calling the Anthropic client with
streaming=Trueandtool_choice={"type": "any"/"tool" }the output returns a tool call but with empty args.This is problematic for a few other reasons beyond no args being returned. For example, quite a few packages rely on the
anthropic-sdk, one of which islangchain-anthropic(ref).Expected response
I would expect that the output includes the inputs/args required for the tool call when streaming.
Reproduction steps
I've added a notebook to highlight some things:
https://gist.github.com/kwnath/f42737c023767d5effdcca20cb5bd0a6
The text was updated successfully, but these errors were encountered: