Invoking the agent asyncrhonously takes a lot more time compared to it's sync method

[helsey33]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain import hub
from langchain.agents import AgentExecutor, create_react_agent
from langchain_community.tools.tavily_search import TavilySearchResults
from clients.llm_client import LLMClient
import os
import time
import asyncio

os.environ["TAVILY_API_KEY"] = "YOUR_API_KEY"

tools = [TavilySearchResults(max_results=1)]
llm = LLMClient().get_genai_chat_v2()
prompt = hub.pull("hwchase17/react")

agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
user_input = "what is the height of the eiffel tower?"


def call_agent_sync():
    start = time.time()
    response = agent_executor.invoke({"input": user_input})
    end = time.time()
    print(f"Sync call took {end - start} seconds")


async def call_agent_async():
    start = time.time()
    response = await agent_executor.ainvoke({"input": user_input})
    end = time.time()
    print(f"Async call took {end - start} seconds")


# call_agent_sync()
asyncio.run(call_agent_async())

Description

The sync call is taking about 12s while the async call is taking a whopping 52s. I understand that async implementations have an overhead but is the overhead really that huge?

Note: we are using llm_client which is a wrapper around OpenAI

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.6.0: Wed Jul 31 20:48:44 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_X86_64
Python Version: 3.11.9 (main, Apr 2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]

Package Information

langchain_core: 0.2.10
langchain: 0.2.6
langchain_community: 0.2.6
langsmith: 0.1.82
langchain_openai: 0.1.7
langchain_text_splitters: 0.2.2
langchainhub: 0.1.21

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph
langserve