Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic 2 compatibility change has broken Playwright's NavigateBackTool, CurrentWebPageTool and ExtractTextTool #26758

Open
5 tasks done
richjames0 opened this issue Sep 22, 2024 · 0 comments
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate

Comments

@richjames0
Copy link

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code, with an OpenAI key specified at the TODO

from langchain_community.tools.playwright.utils import create_sync_playwright_browser
from langchain_community.agent_toolkits import PlayWrightBrowserToolkit
from langchain.agents import create_openai_tools_agent
from langchain_openai import ChatOpenAI
import os
from langchain.schema import SystemMessage
from langchain_core.prompts import (
    HumanMessagePromptTemplate,
    MessagesPlaceholder,
    ChatPromptTemplate,
)

HEADLESS = True

sync_browser = create_sync_playwright_browser(headless=HEADLESS)
toolkit = PlayWrightBrowserToolkit.from_browser(sync_browser=sync_browser)
tools = toolkit.get_tools()
human_prompt = ''
system_prompt = ''
MODEL = "gpt-4o-mini"
SEED = 1337
TEMPERATURE = 0.0

prompt_messages = [
    SystemMessage(content=system_prompt),
    HumanMessagePromptTemplate.from_template(
        template=[  # type:ignore
            {
                "type": "text",
                "text": human_prompt,
            },
            {
                "type": "image_url",
                "image_url": {"url": "data:image/jpeg;base64, {image_data}"},
            },
        ]
    ),
    MessagesPlaceholder("agent_scratchpad"),
]
prompt = ChatPromptTemplate(messages=prompt_messages)
OPEN_AI_KEY = #TODO
os.environ["OPENAI_API_KEY"] = OPEN_AI_KEY
llm = ChatOpenAI(model=MODEL, temperature=TEMPERATURE, seed=SEED)
agent = create_openai_tools_agent(llm, tools, prompt)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain/agents/openai_tools/base.py", line 94, in create_openai_tools_agent
    tools=[convert_to_openai_tool(tool, strict=strict) for tool in tools]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 441, in convert_to_openai_tool
    oai_function = convert_to_openai_function(tool, strict=strict)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 388, in convert_to_openai_function
    oai_function = cast(dict, format_tool_to_openai_function(function))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/_api/deprecation.py", line 179, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/function_calling.py", line 292, in format_tool_to_openai_function
    if tool.tool_call_schema and not is_simple_oai_tool:
       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/tools/base.py", line 457, in tool_call_schema
    return _create_subset_model(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/pydantic.py", line 326, in _create_subset_model
    return _create_subset_model_v2(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venv/iconolist/lib/python3.12/site-packages/langchain_core/utils/pydantic.py", line 270, in _create_subset_model_v2
    field = model.model_fields[field_name]  # type: ignore
            ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 'data'

Description

Note: The following is written for NavigateBackTool (where I first discovered the problem) but is the same for ExtractTextTool and CurrentWebPageTool.

The final line of the code above fails with the error message given. It fails in some sense because the args_schema of the NavigateBackTool claims to support a field 'data' which it actually does not (briefly, the code seems to get a list of properties which can be queried and then query them and there's a mismatch here). My guess is that this is a version mismatch between v1 and v2 of pydantic but I could be wrong (I am not a pydantic expert). I believe this was introduced in PR [0].

Looking at navigate_back.py I can see that the import of BaseModel has changed e.g. from [1] to [2], where [1] was explicitly including the v1 version of BaseModel (albeit from langchain_core) and 2 pulls from pydantic.BaseModel.

Other tools e.g. ClickTool don't have this problem, because they override BaseModel to provide and explicit "Input" class. I have confirmed that creating such a class is a workaround for this issue. FWIW I think there's a strong argument that making this empty set of params explicit and not implicit is better anyway.

Apologies that this isn't the most complete explanation but I hope it's enough for someone in the know to understand what's going in detail much more quickly than it would take me to understand exactly why e.g. 'data' is listed as a field when not actually present :) I have found [3] to be a great place to set a breakpoint to understand this.

[0] #26443
[1] https://api.python.langchain.com/en/latest/_modules/langchain_community/tools/playwright/navigate_back.html#NavigateBackTool
[2] https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/tools/playwright/navigate_back.py
[3] Property method tool_call_schema in VENV_PATH/lib/python3.12/site-packages/langchain_core/tools/base.py

System Info

python -m langchain_core.sys_info

System Information
------------------
> OS:  Linux
> OS Version:  #13-Ubuntu SMP Mon Jul 15 13:40:27 UTC 2024
> Python Version:  3.12.3 (main, Sep 11 2024, 14:17:37) [GCC 13.2.0]

Package Information
-------------------
> langchain_core: 0.3.5
> langchain: 0.3.0
> langchain_community: 0.3.0
> langsmith: 0.1.125
> langchain_openai: 0.2.0
> langchain_text_splitters: 0.3.0

Optional packages not installed
-------------------------------
> langgraph
> langserve

Other Dependencies
------------------
> aiohttp: 3.10.5
> async-timeout: Installed. No version info available.
> dataclasses-json: 0.6.7
> httpx: 0.27.2
> jsonpatch: 1.33
> numpy: 1.26.4
> openai: 1.47.0
> orjson: 3.10.7
> packaging: 24.1
> pydantic: 2.9.2
> pydantic-settings: 2.5.2
> PyYAML: 6.0.2
> requests: 2.32.3
> SQLAlchemy: 2.0.35
> tenacity: 8.5.0
> tiktoken: 0.7.0
> typing-extensions: 4.12.2
@langcarl langcarl bot added the investigate label Sep 22, 2024
@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Sep 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature investigate
Projects
None yet
Development

No branches or pull requests

1 participant