pydantic.v1.error_wrappers.ValidationError: 2 validation errors for SQLDatabaseChain __root__ -> llm instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)

[prateek6786]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import os
# Setting environment variable
os.environ['CURL_CA_BUNDLE'] = ''
import urllib.parse
from langchain_experimental.sql import SQLDatabaseChain
from langchain_community.utilities import SQLDatabase
from transformers import AutoModelForCausalLM

# Example usage
server = '10.112.62.30,1433'
database = 'Trends'
username = 'susantakumar.sahoo'
password = '########'

uri = f"mssql+pyodbc://{username}:{urllib.parse.quote_plus(password)}@{server}/{database}?driver=ODBC+Driver+17+for+SQL+Server"

input_db = SQLDatabase.from_uri(uri)
print(input_db)

llm = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-medium")

print(llm)

db_agent = SQLDatabaseChain(llm=llm,
                            database=input_db,
                            verbose=True)

db_agent.run("how many table name start with word e?")

Description

I am trying to fetch data from sql database using local llm, but while loading local llm it is not working for me. And throwing the below error. please help me know how to fix, or pls suggest which model can i use instead of it and load it, so it work for me

System Info

langchain_experiment - 0.0.57
langchain_community - 0.0.37

[dosubot[bot]]

The error you're encountering is due to the SQLDatabaseChain class expecting an instance that matches the Runnable interface, but it's receiving an AutoModelForCausalLM instance instead. This mismatch leads to the pydantic.v1.error_wrappers.ValidationError.

To fix this, you need to wrap your AutoModelForCausalLM instance in a class that implements the Runnable interface, which is a requirement for the llm parameter in SQLDatabaseChain. The LangChain documentation or source code should provide examples of classes that implement Runnable, guiding you on how to create a compatible wrapper for your model.

If you're considering an alternative model, the LangChain library's SQLDatabaseChain class is designed to work with language models, and examples in the documentation suggest that models from the OpenAI API, such as GPT-3, are commonly used. You might want to explore using an OpenAI model, which should be directly compatible with the SQLDatabaseChain class, avoiding the need for additional wrappers.

Here's a brief example of how you might use an OpenAI model with SQLDatabaseChain:

from langchain_experimental.sql import SQLDatabaseChain
from langchain_community.llms import OpenAI, SQLDatabase

db = SQLDatabase(...)
db_chain = SQLDatabaseChain.from_llm(OpenAI(), db)

This approach should help you bypass the validation error by ensuring compatibility with the Runnable interface expected by SQLDatabaseChain.

Details

Sources

• pydantic.error_wrappers.ValidationError: 2 validation errors for LLMChain
• Can't use SQLdatabasechain with Multipromptchain
• DOC: Quickstart Code Fails for Retrieval Chain
• libs/core/langchain_core/pydantic_v1/main.py
• libs/langchain/langchain/pydantic_v1/init.py
• libs/core/langchain_core/pydantic_v1/init.py
• libs/experimental/langchain_experimental/sql/base.py
• docs/docs/guides/pydantic_compatibility.md
• docs/docs/guides/development/pydantic_compatibility.md
• libs/experimental/langchain_experimental/pydantic_v1/init.py
• libs/community/pyproject.toml
• libs/experimental/pyproject.toml
• libs/community/langchain_community/utilities/sql_database.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[prateek6786]

Hi, using openai will required the api, which i don't want to use. i want to focus on any local llm can you help to know hich model should i use and where from i can get it. @dosu

[dosubot[bot]]

To use a local language model (LLM) with SQLDatabaseChain without relying on external APIs like OpenAI, you'll need to wrap your AutoModelForCausalLM instance in a custom class that implements the Runnable interface required by LangChain. Here's how you can do it:

1. Create a Custom LLM Class: This class should inherit from LangChain's LLM class and wrap the AutoModelForCausalLM instance. It must implement the necessary methods to be compatible with SQLDatabaseChain.

2. Implement Required Methods: Your custom class needs to implement at least the _call method, which takes a prompt as input, generates a response using the model, and returns the response. You'll also need to define _llm_type and _identifying_params properties.

Here's a basic example of such a class:

from transformers import AutoModelForCausalLM, AutoTokenizer
from langchain_core.language_models.llms import LLM

class MyCustomLLM(LLM):
    def __init__(self, model_name: str):
        self.model = AutoModelForCausalLM.from_pretrained(model_name)
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)

    def _call(self, prompt: str, **kwargs):
        input_ids = self.tokenizer.encode(prompt, return_tensors="pt")
        output = self.model.generate(input_ids, max_length=50)
        response = self.tokenizer.decode(output[0], skip_special_tokens=True)
        return response[len(prompt):]  # Return only the generated part

    @property
    def _llm_type(self) -> str:
        return "custom-causal-lm"

You can then use an instance of MyCustomLLM as the llm parameter for SQLDatabaseChain:

llm = MyCustomLLM("microsoft/DialoGPT-medium")
db_agent = SQLDatabaseChain(llm=llm, database=input_db, verbose=True)

This approach allows you to use any local LLM that can be loaded with AutoModelForCausalLM and AutoTokenizer from the transformers library, making it compatible with SQLDatabaseChain without needing an external API.

Details

Sources

• CustomLLM cannot be used to build Chains when using LLM or Runnable
• replace openai
• Please provide tutorials for using other LLM models beside OpenAI.
• docs/docs/modules/model_io/llms/custom_llm.ipynb
• libs/community/langchain_community/llms/bigdl_llm.py
• libs/langchain/langchain/schema/runnable/init.py
• libs/core/langchain_core/runnables/base.py
• libs/core/langchain_core/language_models/init.py
• libs/community/langchain_community/llms/ollama.py
• libs/experimental/langchain_experimental/chat_models/init.py
• libs/experimental/langchain_experimental/sql/base.py
• cookbook/sql_db_qa.mdx
• libs/experimental/langchain_experimental/sql/vector_sql.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[jasonchenTJ]

Hi Dosu, @dosu

I copied your code to warpped a LLM , but got below error:

Traceback (most recent call last):
File "pgdbchain.py", line 20, in
llm = MyCustomLLM("/root/autodl-tmp/LLM/Llama3-Chinese/Llama3-Chinese")
File "/root/customLLM.py", line 7, in init
self.model= AutoModelForCausalLM.from_pretrained(model_name)
File "/root/miniconda3/lib/python3.8/site-packages/pydantic/v1/main.py", line 357, in setattr
raise ValueError(f'"{self.class.name}" object has no field "{name}"')
ValueError: "MyCustomLLM" object has no field "model"

Can you help to fix it?

Thanks
Jason