Compatibility issue with faiss-cpu wheels on arm64 vs x86

[NikhilSehgal123]

Describe the bug
FAISS has different wheels for arm64 chips & Intel/AMD chips. If you vectorise a document using arm64, you should use FAISS to interact with this document on a arm64 machine.

So right now, my mac is arm64 and I vectorised docs on my mac using the arm64 wheel. However in the Azure functions, it attempts to use the Intel wheel which is slightly different. Intel/AMD chips use something called AVX2 support in FAISS (no idea what it actually is but it's different to the way arm64 machines use FAISS).

How can this be solved?
Normalise the wheels across all architectures to solve this compatibility issue problem - happy to help?

[kyamagu]

avx2 is a CPU instruction set. I have no idea what is the binary compatibility to the saved indices across different architectures. Ask the question in the official repository.

[max-addison]

Yep, I have this exact issue. I've deployed to AWS EKS but built the vectorstore on my mac locally and am getting errors. Would be great to get this fixed! @NikhilSehgal123 if you post this issue in the official repository, please post the link here, and i'll +1 and comment! 🙏

[jpzhangvincent]

Ya it would be great to fix this issue soon if possible!

[NikhilSehgal123]

Hey guys @max-addison @jpzhangvincent, I know this issue is causing some pain for many people, so another simple workaround is to just upload your vectorstores to Pinecone or another cloud hosted vector DB to avoid this type of issue.

[Alex-Wenner-FHR]

I experienced this same issue when using LangChain.

here there is the following code:

Import faiss if available, otherwise raise error.
    If FAISS_NO_AVX2 environment variable is set, it will be considered
    to load FAISS with no AVX2 optimization.

When I explicitly did not set FAISS_NO_AVX2 this error subsides. Previously I was experiencing the same thing mentioned in this thread. The faiss index that I am working with today, was created on a windows machine and running successfully on a linux machine. So it seems like loading FAISS with AVX2 optimization resolved the issue for me. Not sure this is helpful to everybody, but hopefully it will help somebody 😄

[aiakubovich]

@Alex-Wenner-FHR

I was trying FAISS_NO_AVX2 but it does not work inside of docker container for some reason (getting TypeError: IndexFlatCodes.add() missing 1 required positional argument: 'x' error). I opened issue: langchain-ai/langchain#8857 . Please let me know if you know why I am getting that error.

[Alex-Wenner-FHR]

@aiakubovich I actually ran into this as well. I am trying to recall what I had done... I know I had troubles with pathing and being able to find the index.faiss in the faiss_index folder. I do believe I had to recreate the Faiss_Index on the same type of machine I was trying to inference... for example, originally it was on Windows and trying to be ran on a linux machine, I had to recreate it on a linux machine to run it on the linux machine.

[aiakubovich]

@Alex-Wenner-FHR Thanks for your response. I thought that FAISS_NO_AVX2 is option that allows to use same embeddings for both Windows and Linux machines. Without this option I indeed have to rerun embeddings on different machines. With this option I unable to run embeddings in Linux container at all. No idea why.

[aboutmydreams]

Same problem to run on linux but not on m2/m1 mac

langchain-ai/chat-langchain#61