openai: fix json_schema mode in AzureChatOpenAI.with_structured_output

[Skar0]

This PR fixes an issue with json_schema mode in AzureChatOpenAI.with_structured_output.

Following the release of structured output support in Azure OpenAI, I was able to test #25591 where the main proposed change is to use BaseChatOpenAI.with_structured_output and not override it in AzureChatOpenAI.with_structured_output like currently implemented. When testing json_schema mode with those proposed changes, I was getting ValueError: Structured Output response does not have a 'parsed' field nor a 'refusal' field..

After some investigation, here is an analysis of the issue:

• In BaseChatOpenAI._create_chat_result, the parsed and refusal properties of the completion generated by BaseChatOpenAI._generate are correctly added as additional_kwargs to the generation.
• In that case, BaseChatOpenAI._create_chat_result is called with the argument response of type openai.BaseModel.
• However, AzureChatOpenAI._create_chat_result overrides BaseChatOpenAI._create_chat_result like so
  

  langchain/libs/partners/openai/langchain_openai/chat_models/azure.py

  Lines 985 to 993 in de97d50

  	if not isinstance(response, dict):
  	response = response.model_dump()
  	for res in response["choices"]:
  	if res.get("finish_reason", None) == "content_filter":
  	raise ValueError(
  	"Azure has not provided the response due to a content filter "
  	"being triggered"
  	)
  	chat_result = super()._create_chat_result(response, generation_info)

• BaseChatOpenAI._create_chat_result is thus called with the argument response of type dict (following response.model_dump()).
• It follows that the condition below in the implementation of BaseChatOpenAI._create_chat_result is not satisfied and thus parsed and refusal are not added to the generation.
  

  langchain/libs/partners/openai/langchain_openai/chat_models/base.py

  Lines 719 to 726 in de97d50

  	if isinstance(response, openai.BaseModel) and getattr(
  	response, "choices", None
  	):
  	message = response.choices[0].message # type: ignore[attr-defined]
  	if hasattr(message, "parsed"):
  	generations[0].message.additional_kwargs["parsed"] = message.parsed
  	if hasattr(message, "refusal"):
  	generations[0].message.additional_kwargs["refusal"] = message.refusal

I propose to change the implementation of that check so that it works on the response_dict that is created anyways in BaseChatOpenAI._create_chat_result so that the parsed and refusal properties of the completion are correctly added as additional_kwargs to the generation whatever the type of response.

@baskaryan I believe it would be valuable for you to review to avoid any regression and confirm that this PR should be merged at the same time as #25591 to avoid introducing a bug.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		Sep 5, 2024 0:02am