Applying Multiple 'OR' Filters with FAISS Retriever

[phoebeeeeeeeeee]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

results_with_scores = db.similarity_search_with_score("foo", filter=dict(page=1))

Description

Hello everyone,

I'm looking for some guidance on using the FAISS retriever to handle multiple filters for document retrieval.
The official documentation indicates that we can apply a single filter parameter to narrow down our search, as demonstrated by:

results_with_scores = db.similarity_search_with_score("foo", filter=dict(page=1))

However, I'm unsure how to proceed when I need to include multiple filter values and a match with any one of those values would suffice.

Additionally, I've experimented with other vector stores and encountered issues where certain methods found on Internet were not effective. It would be great if you could share any examples that illustrate how to implement this type of 'any-match' filtering.

Thank you so much for your help!

System Info

Python 3.9.7
langchain latest version

[dosubot[bot]]

Hey @phoebeeeeeeeeee, great to see you diving into the depths of LangChain again! How's everything going on your end?

To implement multiple 'any-match' filters for document retrieval using the FAISS retriever in LangChain, you can use the $or operator in the filter argument. This operator allows you to specify multiple conditions, and it will return documents that match any of the specified conditions.

Here's an example of how you can use the $or operator to apply multiple filters:

results_with_scores = db.similarity_search_with_score("foo", filter={'$or': [{'page': {'$eq': 1}}, {'page': {'$eq': 2}}]})

In this example, the similarity_search_with_score method will return documents that have a page value of either 1 or 2.

If you want to apply filters based on a list of values, you can use the $in operator. Here's an example:

results_with_scores = db.similarity_search_with_score("foo", filter={'page': {'$in': [1, 2, 3]}})

In this example, the similarity_search_with_score method will return documents that have a page value of either 1, 2, or 3.

Please note that the $eq operator checks for equality, and the $in operator checks if a value is in a list of values.

I hope this helps! If you have any other questions, feel free to ask.

Sources

• Issue: Create way to do filters using VectorStore
• How to pass filter down to Chroma db when using ConversationalRetrievalChain
• can't seem to add filters in ConversationalRetrievalChain
• libs/langchain/langchain/vectorstores/singlestoredb.py

Sources

• Multiple Filters using Chroma().as_retriever
• docs/docs/integrations/vectorstores/faiss.ipynb
• libs/community/langchain_community/vectorstores/faiss.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[YannLex]

filter in FAISS is post-filter