Getting only one matching result from AWS kendra retriver

[vinayburam]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class CustomAmazonKendraRetriever(AmazonKendraRetriever):
    def _kendra_query(self, query: str) -> Sequence[QueryResultItem]:
        kendra_kwargs = {
            "IndexId": self.index_id,
            "QueryText": query.strip()[0:999],
            "PageSize": self.top_k,
        }
        print("Checking att")
        if self.attribute_filter is not None:
            print("Attribut filter: ",self.attribute_filter)
            kendra_kwargs["AttributeFilter"] = self.attribute_filter
        if self.document_relevance_override_configurations is not None:
            kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
                self.document_relevance_override_configurations
            )
        if self.user_context is not None:
            kendra_kwargs["UserContext"] = self.user_context

        response = self.client.query(**kendra_kwargs)
        q_result = QueryResult.parse_obj(response)
        print("q_result: ",q_result)
        return [QueryResultItem(**item.dict()) for item in q_result.ResultItems]

Description

I am using above code. tried to change the topk value to 10. but still getting only one matching result

below is the response
q_result: ResultItems=[QueryResultItem(Id='e64b9d48-7d27-4ce7-b81d-ff7dda60b686-993724c6-54b2-4471-b4f4-1d4788c0b0e9', DocumentId='s3://automatictenderaig/GBC_Master_09122024.json', DocumentURI='https://s3.us-east-1.amazonaws.com/automatictenderaig/GBC_Master_09122024.json', DocumentAttributes=[DocumentAttribute(Key='_data_source_id', Value=DocumentAttributeValue(DateValue=None, LongValue=None, StringListValue=None, StringValue='1bf6a354-7aef-471e-b5b9-9ce733e9d085')), DocumentAttribute(Key='_source_uri', Value=DocumentAttributeValue(DateValue=None, LongValue=None, StringListValue=None, StringValue='https://s3.us-east-1.amazonaws.com/automatictenderaig/GBC_Master_09122024.json'))], ScoreAttributes={'ScoreConfidence': 'MEDIUM'}, DocumentTitle=TextWithHighLights(Text='GBC_Master_09122024', Highlights=[]), FeedbackToken='AYADeIJcFH7jB6tjAgMUGESML3wAXwABABVhd3MtY3J5cHRvLXB1YmxpYy1rZXkAREFzeHl3VFVTMi9idEk4cStrSk4xdXkzejlPNzliRXNHNUpYSnFLd0lDeS91Zi9PaUVGR3ZzWC80eXlJL2pYSUFPUT09AAEAB2F3cy1rbXMAS2Fybjphd3M6a21zOnVzLWVhc3QtMTo3OTQyODk5MjcwNzA6a2V5LzA4YjVkYTRmLWQyOWEtNDU3Mi04OTAwLTRkZjMzN2VjYzljYwC4AQIBAHjHtYTqclWDjf5WL-m_xsZq1b9mncgz9KNLQr3u2EEmpAF9BOgPaFikB1K59u69YJtYAAAAfjB8BgkqhkiG9w0BBwagbzBtAgEAMGgGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMo787muuN8hsCo4ivAgEQgDu7HIdxY95c0R5QAN6eNzoDRL_j7tLRAw2NxBXwypj0Wh3MuHPLMNoOqgOgP6J8y61MIIUXWMl42YmUQgIAAAAADAAAEAAAAAAAAAAAAAAAAABFQ3n4_UDS_0_IkyM9elh-___wAAAAEAAAAAAAAAAAAAAAEAAAF2UqgbpLwgYAi497P0Fba4xD2-CJMknK7y27dhyUZNDbBsd6uoIlQj1TySbGxwGucjaaPmdhDbXsexzm-K30A7aWUFIVF0QhpHSahD49AXViTGQOHBd3YJeJB1lYR6WLBpJzLDz2Q0ZScCjsgwjOiYCTq5ELbBSZCxMzDFcv_rX6FhSfkZTj0Q431xzUOc99zBDgDi1hPjyHVKnTOj5Q5sxh-QYeD3AAQDjMi_R3bzzJvXmBrcSfReIBdrx8cufVz9JUq0SVp22ucDrqUGewIbrZG6s_0L5-GY-vlHKVkkEfyb-wT_yn_Su_BEZtezdIxpHjfNNiwt26w-Uv-rtHQSOmHX5kwWX10WOWJjSik-azhlMH6YIQzLbXJVNCSdPs6OCBTnwmn96YkTBjRsqq8bFnEq-wQKmhsV11xfUntA7K0LSc5mxU53VfQ_L_sZuXQFusyzvMojb8JvaN0U9QBDPTevVT-JhRI3ccKpHsdKIC2L_2LNIlsrVtgkv6vd7lc4rnLFTbAGcwZQIwBm5k2qnDlCndcl1DqCqAuGbN-5sdq0FZidr3Rrrlsf6eIXcUE9doHKHSGEymBspzAjEAg2WTKBWnlQrjXHk15Ok5Vhr0te2c1MBkN-kmRyhXjGvb6NC-XTQ0kqep4wUWkeuR.e64b9d48-7d27-4ce7-b81d-ff7dda60b686-993724c6-54b2-4471-b4f4-1d4788c0b0e9', Format='TEXT', Type='DOCUMENT', AdditionalAttributes=[], DocumentExcerpt=TextWithHighLights(Text='...ID": null\n },\n {\n "PRODUCT NAME": "DC Motors",\n "PRODUCT DESCRIPTION": "Slotless Brushless DC Motor",\n "PREFERRED SUPPLIER": "DINGS",\n "SUPPLIER CONTACT": "Daniel Jang",\n "SUPPLIER EMAIL": "[email protected]?cc=mohamed...', Highlights=[{'BeginOffset': 35, 'EndOffset': 39, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 43, 'EndOffset': 45, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 46, 'EndOffset': 52, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 102, 'EndOffset': 104, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 105, 'EndOffset': 110, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 128, 'EndOffset': 136, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 153, 'EndOffset': 161, 'TopAnswer': False, 'Type': 'STANDARD'}, {'BeginOffset': 192, 'EndOffset': 200, 'TopAnswer': False, 'Type': 'STANDARD'}]))] TotalNumberOfResults=1 QueryId='e64b9d48-7d27-4ce7-b81d-ff7dda60b686' FacetResults=[{'DocumentAttributeKey': '_data_source_id', 'DocumentAttributeValueType': 'STRING_VALUE', 'DocumentAttributeValueCountPairs': [{'DocumentAttributeValue': {'StringValue': '1bf6a354-7aef-471e-b5b9-9ce733e9d085'}, 'Count': 1}]}] ResponseMetadata={'RequestId': '2dbc2392-db69-4dca-b617-9faf3544accd', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '2dbc2392-db69-4dca-b617-9faf3544accd', 'content-type': 'application/x-amz-json-1.1', 'content-length': '3220', 'date': 'Fri, 13 Sep 2024 11:45:01 GMT'}, 'RetryAttempts': 0}

System Info

NA

[dosubot[bot]]

You are getting only one matching result from AWS Kendra Retriever despite setting the top_k value to 10 because the _kendra_query method in your CustomAmazonKendraRetriever class uses the query API directly. The query API might not return the number of results specified by top_k if there are not enough relevant documents.

Here is the relevant code snippet from the AmazonKendraRetriever class that shows how it handles the retrieval process:

def _kendra_query(self, query: str) -> Sequence[ResultItem]:
    kendra_kwargs = {
        "IndexId": self.index_id,
        "QueryText": query.strip()[0:999],
        "PageSize": self.top_k,
    }
    if self.attribute_filter is not None:
        kendra_kwargs["AttributeFilter"] = self.attribute_filter
    if self.document_relevance_override_configurations is not None:
        kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
            self.document_relevance_override_configurations
        )
    if self.user_context is not None:
        kendra_kwargs["UserContext"] = self.user_context

    response = self.client.retrieve(**kendra_kwargs)
    r_result = RetrieveResult.parse_obj(response)
    if r_result.ResultItems:
        return r_result.ResultItems

    # Retrieve API returned 0 results, fall back to Query API
    response = self.client.query(**kendra_kwargs)
    q_result = QueryResult.parse_obj(response)
    return q_result.ResultItems

In this code, the retrieve API is used first, and if it returns 0 results, it falls back to the query API. If the retrieve API returns fewer results than top_k, it will not attempt to get more results from the query API ^[1].

To address this, you might want to ensure that your CustomAmazonKendraRetriever class also attempts to use the retrieve API first before falling back to the query API. This way, you can maximize the number of results returned.

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

The discrepancy between the results from the retrieve API and the query API in your CustomAmazonKendraRetriever class is likely due to the differences in how the RetrieveResultItem and QueryResultItem classes handle the document title and excerpt.

The RetrieveResultItem class uses the DocumentTitle and Content fields directly:

class RetrieveResultItem(ResultItem):
    """Retrieve API result item."""

    DocumentTitle: Optional[str]
    """The document title."""
    Content: Optional[str]
    """The content of the item."""

    def get_title(self) -> str:
        return self.DocumentTitle or ""

    def get_excerpt(self) -> str:
        return self.Content or ""

In contrast, the QueryResultItem class uses DocumentTitle and DocumentExcerpt, which are instances of TextWithHighLights, and also considers AdditionalAttributes:

class QueryResultItem(ResultItem):
    """Query API result item."""

    DocumentTitle: TextWithHighLights
    """The document title."""
    FeedbackToken: Optional[str]
    """Identifies a particular result from a particular query."""
    Format: Optional[str]
    """
    If the Type is ANSWER, then format is either:
        * TABLE: a table excerpt is returned in TableExcerpt;
        * TEXT: a text excerpt is returned in DocumentExcerpt.
    """
    Type: Optional[str]
    """Type of result: DOCUMENT or QUESTION_ANSWER or ANSWER"""
    AdditionalAttributes: Optional[List[AdditionalResultAttribute]] = []
    """One or more additional attributes associated with the result."""
    DocumentExcerpt: Optional[TextWithHighLights]
    """Excerpt of the document text."""

    def get_title(self) -> str:
        return self.DocumentTitle.Text

    def get_attribute_value(self) -> str:
        if not self.AdditionalAttributes:
            return ""
        if not self.AdditionalAttributes[0]:
            return ""
        else:
            return self.AdditionalAttributes[0].get_value_text()

    def get_excerpt(self) -> str:
        if (
            self.AdditionalAttributes
            and self.AdditionalAttributes[0].Key == "AnswerText"
        ):
            excerpt = self.get_attribute_value()
        elif self.DocumentExcerpt:
            excerpt = self.DocumentExcerpt.Text
        else:
            excerpt = ""

        return excerpt

    def get_additional_metadata(self) -> dict:
        additional_metadata = {"type": self.Type}
        return additional_metadata

The QueryResultItem class provides a more detailed and structured way to extract the title and excerpt, which might be why the results from the query API are more accurate. The retrieve API might not be populating the DocumentTitle and Content fields in the same way, leading to unexpected results ^[1].

To continue talking to Dosu, mention @dosu.

[vinayburam]

any solution for this @dosu

[dosubot[bot]]

Yes, there is a solution to ensure that the retrieve API in the CustomAmazonKendraRetriever class returns accurate results similar to the query API. The provided code already includes a mechanism to fall back to the query API if the retrieve API returns 0 results. This is implemented in the _kendra_query method.

Here is the relevant part of the code:

class CustomAmazonKendraRetriever(AmazonKendraRetriever):
    def _kendra_query(self, query: str) -> Sequence[ResultItem]:
        kendra_kwargs = {
            "IndexId": self.index_id,
            "QueryText": query.strip()[0:999],
            "PageSize": self.top_k,
        }
        if self.attribute_filter is not None:
            kendra_kwargs["AttributeFilter"] = self.attribute_filter
        if self.document_relevance_override_configurations is not None:
            kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
                self.document_relevance_override_configurations
            )
        if self.user_context is not None:
            kendra_kwargs["UserContext"] = self.user_context

        response = self.client.retrieve(**kendra_kwargs)
        r_result = RetrieveResult.parse_obj(response)
        if r_result.ResultItems:
            return r_result.ResultItems

        # Retrieve API returned 0 results, fall back to Query API
        response = self.client.query(**kendra_kwargs)
        q_result = QueryResult.parse_obj(response)
        return q_result.ResultItems

This method first attempts to retrieve results using the retrieve API. If no results are returned (r_result.ResultItems is empty), it falls back to using the query API to get the results. This ensures that the retrieve API returns accurate results similar to the query API ^[1].

Additionally, you can improve the accuracy of the retrieve API by setting additional parameters and configurations, such as max_documents, get_extractive_answers, and max_extractive_answer_count. These configurations can help fine-tune the retrieval process to better match your specific use case ^[2][3].

To continue talking to Dosu, mention @dosu.

[vinayburam]

it is the same code @dosu

[dosubot[bot]]

The discrepancy between the retrieve API and the query API results is due to their different implementations and how they handle search results. The retrieve API is designed to return relevant passages or text excerpts, while the query API can return a broader range of results, including suggested answers, matching FAQs, and documents with excerpts.

To ensure that the retrieve API returns results similar to the query API, you can adjust the parameters and configurations used in the retrieve API call. Here is the updated code for your CustomAmazonKendraRetriever class:

from langchain_community.retrievers.kendra import RetrieveResult, QueryResult, ResultItem

class CustomAmazonKendraRetriever(AmazonKendraRetriever):
    def _kendra_query(self, query: str) -> Sequence[ResultItem]:
        kendra_kwargs = {
            "IndexId": self.index_id,
            "QueryText": query.strip()[0:999],
            "PageSize": self.top_k,
        }
        if self.attribute_filter is not None:
            kendra_kwargs["AttributeFilter"] = self.attribute_filter
        if self.document_relevance_override_configurations is not None:
            kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
                self.document_relevance_override_configurations
            )
        if self.user_context is not None:
            kendra_kwargs["UserContext"] = self.user_context

        response = self.client.retrieve(**kendra_kwargs)
        r_result = RetrieveResult.parse_obj(response)
        if r_result.ResultItems:
            return r_result.ResultItems

        # Retrieve API returned 0 results, fall back to Query API
        response = self.client.query(**kendra_kwargs)
        q_result = QueryResult.parse_obj(response)
        return q_result.ResultItems

This code ensures that the retrieve API is used first, and if it returns 0 results, it falls back to the query API. The attribute filters and other configurations are included in the kendra_kwargs dictionary if they are not None ^[1].

To continue talking to Dosu, mention @dosu.

[vinayburam]

can we priotorize the result of query api first and then combine this with retrive api @dosu

[dosubot[bot]]

Yes, you can prioritize the result of the query API first and then combine this with the retrieve API. The _kendra_query method in the provided code already implements a similar logic but in reverse order. To prioritize the query API first, you can modify the method as follows:

from langchain_community.retrievers.kendra import RetrieveResult, QueryResult, ResultItem

class CustomAmazonKendraRetriever(AmazonKendraRetriever):
    def _kendra_query(self, query: str) -> Sequence[ResultItem]:
        kendra_kwargs = {
            "IndexId": self.index_id,
            "QueryText": query.strip()[0:999],
            "PageSize": self.top_k,
        }
        if self.attribute_filter is not None:
            kendra_kwargs["AttributeFilter"] = self.attribute_filter
        if self.document_relevance_override_configurations is not None:
            kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
                self.document_relevance_override_configurations
            )
        if self.user_context is not None:
            kendra_kwargs["UserContext"] = self.user_context

        # First, use the Query API
        response = self.client.query(**kendra_kwargs)
        q_result = QueryResult.parse_obj(response)
        if q_result.ResultItems:
            return q_result.ResultItems

        # If Query API returns 0 results, fall back to Retrieve API
        response = self.client.retrieve(**kendra_kwargs)
        r_result = RetrieveResult.parse_obj(response)
        return r_result.ResultItems

This method first calls the query API and if it returns no results, it then calls the retrieve API ^[1].

Additionally, if you are dealing with multiple queries, you can use a query analysis technique to generate and run all queries, then combine the results. Here is an example using LangChain:

@chain
async def custom_chain(question):
    response = await query_analyzer.ainvoke(question)
    docs = []
    for query in response.queries:
        new_docs = await retriever.ainvoke(query)
        docs.extend(new_docs)
    # You probably want to think about reranking or deduplicating documents here
    # But that is a separate topic
    return docs

This approach allows you to handle multiple queries asynchronously, combining the results efficiently ^[2][3].

To continue talking to Dosu, mention @dosu.

[vinayburam]

@chain
async def custom_chain(question):
response = await query_analyzer.ainvoke(question)
docs = []
for query in response.queries:
new_docs = await retriever.ainvoke(query)
docs.extend(new_docs)
# You probably want to think about reranking or deduplicating documents here
# But that is a separate topic
return docs

where i can use this in my code @dosu

[dosubot[bot]]

You can use the custom_chain function in your code to handle multiple queries asynchronously and combine the results. This function is particularly useful when you need to analyze a question, generate multiple queries, and retrieve documents for each query. Here is an example of how you can integrate the custom_chain function into your existing code:

1. Define the custom_chain function:
   Ensure you have the custom_chain function defined as follows:

@chain
async def custom_chain(question):
    response = await query_analyzer.ainvoke(question)
    docs = []
    for query in response.queries:
        new_docs = await retriever.ainvoke(query)
        docs.extend(new_docs)
    # You probably want to think about reranking or deduplicating documents here
    # But that is a separate topic
    return docs

2. Invoke the custom_chain function:
   You can invoke the custom_chain function in your main code where you need to process a question and retrieve documents. Here is an example:

async def main():
    question = "Your question here"
    documents = await custom_chain(question)
    # Process the documents as needed
    for doc in documents:
        print(doc)

# Run the main function
import asyncio
asyncio.run(main())

3. Ensure all dependencies are set up:
   Make sure you have the necessary components like query_analyzer and retriever properly configured in your code. Here is an example setup:

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI
from langchain_community.retrievers.kendra import AmazonKendraRetriever

# Setup query_analyzer
system = """You are an expert at converting user questions into database queries. \
You have access to a database of tutorial videos about a software library for building LLM-powered applications. \
Given a question, return a list of database queries optimized to retrieve the most relevant results.

If there are acronyms or words you are not familiar with, do not try to rephrase them."""
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        ("human", "{question}"),
    ]
)
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
structured_llm = llm.with_structured_output(Search)
query_analyzer = {"question": RunnablePassthrough()} | prompt | structured_llm

# Setup retriever
retriever = CustomAmazonKendraRetriever(
    index_id="your_index_id",
    region_name="your_region_name",
    credentials_profile_name="your_profile_name",
    top_k=10,
    attribute_filter={"EqualsTo": {"Key": "DocumentType", "Value": {"StringValue": "FAQ"}}},
    document_relevance_override_configurations=[{"Name": "Relevance", "ValueImportanceMap": {"Title": 1}}],
    user_context={"Token": "user-token"},
    min_score_confidence=0.5
)

By following these steps, you can effectively use the custom_chain function in your code to handle multiple queries and combine the results ^[1][2].

To continue talking to Dosu, mention @dosu.

[vinayburam]

I am Using below code.
import json
from typing import Sequence, Dict, Any
from langchain_community.retrievers.kendra import AmazonKendraRetriever, QueryResult, QueryResultItem,RetrieveResult
from langchain.chains import ConversationalRetrievalChain
from langchain.prompts import PromptTemplate
from langchain_openai import AzureChatOpenAI
from langchain.chains.conversation.memory import ConversationBufferMemory
import os
from prompts import prompt_template1

class CustomAmazonKendraRetriever(AmazonKendraRetriever):
def _kendra_query(self, query: str) -> Sequence[QueryResultItem]:
kendra_kwargs = {
"IndexId": self.index_id,
"QueryText": query.strip()[0:999],
"PageSize": self.top_k,
}
print("Checking att")
if self.attribute_filter is not None:
print("Attribut filter: ",self.attribute_filter)
kendra_kwargs["AttributeFilter"] = self.attribute_filter
if self.document_relevance_override_configurations is not None:
kendra_kwargs["DocumentRelevanceOverrideConfigurations"] = (
self.document_relevance_override_configurations
)
if self.user_context is not None:
kendra_kwargs["UserContext"] = self.user_context
#Addidng retrive query api
#response = self.client.retrieve(**kendra_kwargs)
#r_result = RetrieveResult.parse_obj(response)
#if r_result.ResultItems:
#return r_result.ResultItems

    response = self.client.query(**kendra_kwargs)
    q_result = QueryResult.parse_obj(response)
    print("q_result: ",q_result)
    return [QueryResultItem(**item.dict()) for item in q_result.ResultItems]

def build_chain():
region = os.environ["REGION"]
kendra_index_id = os.environ["KENDRA_INDEX"]

#llm = OpenAI(batch_size=5, temperature=0, max_tokens=300)

The base URL for your Azure OpenAI resource. e.g. "https://.openai.azure.com"

#gpt 4 config
azure_endpoint = os.environ["azure_endpoint"]

API version e.g. "2023-07-01-preview"

#openai_api_version="2024-03-01-preview"
openai_api_version="2024-03-01-preview"

The name of your Azure OpenAI deployment chat model. e.g. "gpt-35-turbo-0613"

#deployment_name="gpt-35-turbo"
deployment_name="gpt-4o"

The API key for your Azure OpenAI resource.

openai_api_key = os.environ["openai_api_key"]

This is set to azure

openai_api_type="azure"

llm = AzureChatOpenAI(
azure_endpoint=azure_endpoint,
openai_api_version=openai_api_version,
deployment_name=deployment_name,
openai_api_key=openai_api_key,
openai_api_type=openai_api_type,
)

#retriever = AmazonKendraRetriever(index_id=kendra_index_id, region_name=region)
data_source_id = '1bfd085'
retriever = CustomAmazonKendraRetriever(index_id=kendra_index_id, top_k=3,
attribute_filter = {'AndAllFilters':
[
{"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "en"}}},
{"EqualsTo": {"Key": "_data_source_id","Value": {"StringValue": data_source_id}}}

        ]
        }


) 

'''prompt_template = """
The following is a friendly conversation between a human and an AI.
The chatbot is talkative and provides lots of specific details from its context.
If the chatbot does not know the answer to a question, it suggests some user some products based on the given user input and context
{context}

Instruction: Based on the above documents, provide a below answer for, {question}
Name in the document is supplier name
Decscription column in the document can contain product name as well as description
Tax_title,Supplier name(Name),description, Primary Category Contact, Alternate Category Contact, Global Category Cluster Contact
Add the Tax_title as category in output
Do not add additional information in the response.

            Only suggest some topics to search from the fetched content to the user if there is no relavant result. 
            Do not suggest from internet. Always show this information in text
            Output should be in a proper HTML table format with CSS border styling for each column.

Solution:"""
'''
prompt_template = prompt_template1
PROMPT = PromptTemplate(
template=prompt_template, input_variables=["context", "question"]
)

condense_qa_template = """
Given the following conversation and a follow up question, rephrase the follow up question
to be a standalone question.

Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:"""
standalone_question_prompt = PromptTemplate.from_template(condense_qa_template)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True, input_key='question', output_key='answer')
#print("Convversationbuffermemeory:",memory)
qa = ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=retriever,
condense_question_prompt=standalone_question_prompt,
return_source_documents=True,
memory = memory,
combine_docs_chain_kwargs={"prompt":PROMPT})

return qa

def run_chain(chain, prompt: str, history=[]):
print("History from run chain function",history)
print("User Input prompt: ",prompt)
return chain({"question": prompt, "chat_history": history})

Help me to add below technique in above code
you can use a query analysis technique to generate and run all queries @dosu

[dosubot[bot]]

To integrate a query analysis technique to generate and run all queries in your existing code, you can follow these steps:

1. Define the custom_chain function:
   This function will handle multiple queries asynchronously and combine the results.

from langchain_core.runnables import chain

@chain
async def custom_chain(question):
    response = await query_analyzer.ainvoke(question)
    docs = []
    for query in response.queries:
        new_docs = await retriever.ainvoke(query)
        docs.extend(new_docs)
    # You probably want to think about reranking or deduplicating documents here
    # But that is a separate topic
    return docs

2. Set up the query_analyzer:
   Configure the query_analyzer to generate sub-queries from the main question.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_openai import ChatOpenAI
from langchain_core.runnables import RunnablePassthrough
from langchain_core.pydantic_v1 import BaseModel, Field
from typing import List, Optional

class Search(BaseModel):
    query: str = Field(..., description="Primary similarity search query applied to video transcripts.")
    sub_queries: List[str] = Field(default_factory=list, description="List of sub-queries.")
    publish_year: Optional[int] = Field(None, description="Year video was published")

system = """You are an expert at converting user questions into database queries. \
You have access to a database of tutorial videos about a software library for building LLM-powered applications. \
Given a question, return a list of database queries optimized to retrieve the most relevant results.

If there are acronyms or words you are not familiar with, do not try to rephrase them."""
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        MessagesPlaceholder("examples", optional=True),
        ("human", "{question}"),
    ]
)
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)
structured_llm = llm.with_structured_output(Search)
query_analyzer = {"question": RunnablePassthrough()} | prompt | structured_llm

3. Modify the build_chain function:
   Integrate the custom_chain function into your existing build_chain function.

def build_chain():
    region = os.environ["REGION"]
    kendra_index_id = os.environ["KENDRA_INDEX"]

    azure_endpoint = os.environ["azure_endpoint"]
    openai_api_version = "2024-03-01-preview"
    deployment_name = "gpt-4o"
    openai_api_key = os.environ["openai_api_key"]
    openai_api_type = "azure"

    llm = AzureChatOpenAI(
        azure_endpoint=azure_endpoint,
        openai_api_version=openai_api_version,
        deployment_name=deployment_name,
        openai_api_key=openai_api_key,
        openai_api_type=openai_api_type,
    )

    data_source_id = '1bfd085'
    retriever = CustomAmazonKendraRetriever(
        index_id=kendra_index_id, top_k=3,
        attribute_filter={
            'AndAllFilters': [
                {"EqualsTo": {"Key": "_language_code", "Value": {"StringValue": "en"}}},
                {"EqualsTo": {"Key": "_data_source_id", "Value": {"StringValue": data_source_id}}}
            ]
        }
    )

    prompt_template = prompt_template1
    PROMPT = PromptTemplate(
        template=prompt_template, input_variables=["context", "question"]
    )

    condense_qa_template = """
    Given the following conversation and a follow up question, rephrase the follow up question 
    to be a standalone question.

    Chat History:
    {chat_history}
    Follow Up Input: {question}
    Standalone question:"""
    standalone_question_prompt = PromptTemplate.from_template(condense_qa_template)
    memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True, input_key='question', output_key='answer')

    qa = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=retriever,
        condense_question_prompt=standalone_question_prompt,
        return_source_documents=True,
        memory=memory,
        combine_docs_chain_kwargs={"prompt": PROMPT}
    )

    return qa

4. Invoke the custom_chain function:
   Use the custom_chain function in your main code to process a question and retrieve documents.

async def main():
    question = "Your question here"
    documents = await custom_chain(question)
    # Process the documents as needed
    for doc in documents:
        print(doc)

# Run the main function
import asyncio
asyncio.run(main())

By following these steps, you can effectively integrate a query analysis technique to generate and run all queries in your existing code, combining the results efficiently ^[1][2][3].

To continue talking to Dosu, mention @dosu.