Summary:
Pull Request resolved: facebookresearch#3317

libraft packages are first published in rapidsai-nightly and moved to rapidsai after release, at which point they're removed from rapidsai-nightly

In this diff we enable both channels with a preference to rapidsai (since it's before rapidsai-nightly on the command line).

Reviewed By: mlomeli1

Differential Revision: D55310143

fbshipit-source-id: b85e0fda86a442f435d985ace1d7eb37209c74e1

Summary:
Pull Request resolved: facebookresearch#3312

as the [#issue3258](facebookresearch#3258) mentioned, the IVFPQFastScan should have same decoding result as IVFPQ. However, current result is not as expected.

In this PR/Diff, we are going to fix the decoding function

Reviewed By: mdouze

Differential Revision: D55264781

fbshipit-source-id: dfdae9eabceadfc5a3ebb851930d71ce3c1c654d

Summary:
In the facebookresearch#3278, we to find_package to link to GTest. However, it needs to have googletest to build independently. Not everyone builds their googletest locally first. In this diff, we still try to build googletest from source and combine find_package together.

Pull Request resolved: facebookresearch#3319

Test Plan:
STEP 1: Install deps
```
conda install -y -q python=3.11 cmake make swig=4.0.2 mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64
```

STEP2: Compile

```
cmake -B build \
      -DBUILD_TESTING=ON \
      -DBUILD_SHARED_LIBS=ON \
      -DFAISS_ENABLE_GPU=OFF \
      -DFAISS_ENABLE_RAFT=OFF \
      -DFAISS_OPT_LEVEL=avx2 \
      -DFAISS_ENABLE_C_API=ON \
      -DPYTHON_EXECUTABLE=$(which python) \
      -DCMAKE_BUILD_TYPE=Release \
      -DBLA_VENDOR=Intel10_64_dyn \
      -DCMAKE_CUDA_FLAGS="-gencode arch=compute_75,code=sm_75" \
      .
```

Reviewed By: algoriddle

Differential Revision: D55358059

Pulled By: junjieqi

fbshipit-source-id: 95ad4a745238b88b438728de64173f99d3d50dbe

Summary:
Fixes problem when compiling OpenBLAS with INTERFACE64=1 (64-bit integers).

Pull Request resolved: facebookresearch#3322

Reviewed By: algoriddle

Differential Revision: D55469397

Pulled By: mdouze

fbshipit-source-id: 14d916fb074f6ea0f591e0324bb7b8674a624473

Summary:
Currently, faiss can't build with swig version > 4.2.x. As the facebookresearch#3239 mentioned.  Swig removed the support for 32bit swig/swig@9fb3a49. So SWIGTYPE_p_unsigned_long_long isn't supported any more. In this diff, we are going to remove the unsupported type from Faiss swig.

Pull Request resolved: facebookresearch#3315

Test Plan:
STEP 1: create a new conda env
```
conda create --name faiss_swig
conda activate faiss_swig
```

STEP 2: install dependecies from conda-forge
```
conda install -y -q python=3.11 cmake make swig mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64=2.28 -c conda-forge
```

STEP 3: CMAKE

```
cmake -B build \
      -DBUILD_TESTING=ON \
      -DBUILD_SHARED_LIBS=ON \
      -DFAISS_ENABLE_GPU=OFF \
      -DFAISS_ENABLE_RAFT=OFF \
      -DFAISS_OPT_LEVEL=avx512 \
      -DFAISS_ENABLE_C_API=ON \
      -DPYTHON_EXECUTABLE=$(which python) \
      -DCMAKE_BUILD_TYPE=Release \
      -DBLA_VENDOR=Intel10_64_dyn \
      -DCMAKE_CUDA_FLAGS="-gencode arch=compute_75,code=sm_75" \
      .
```

STEP 4: build
```
make -C build -j faiss && make -C build -j swigfaiss
```
<img width="876" alt="Screenshot 2024-03-25 at 12 24 16 AM" src="https://github.com/facebookresearch/faiss/assets/8333898/918f0caf-398a-4361-989f-93ff547cf2b2">

Reviewed By: algoriddle

Differential Revision: D55304004

Pulled By: junjieqi

fbshipit-source-id: e958009dc637aa33b0e1a574a16a846a4abb1525

Summary:
AVX-512 implementation for PQFastScan for QBS.
For local benchmarks on 4th gen Xeon, the QPS is up to 10% higher, mostly for a single query case. But as far as I remember, production cases would show higher performance improvements.

* Baseline `benchs/bench_ivf_fastscan_single_query.py` (sift1M): https://gist.github.com/alexanderguzhva/c9cde2cb5e9c7675f429623e6faa9fbf
* Candidate `benchs/bench_ivf_fastscan_single_query.py` (sift1M): https://gist.github.com/alexanderguzhva/4e8530073a108f73771d38e55bc45b17
* Baseline `benchs/bench_ivf_fastscan.py` (sift1M): https://gist.github.com/alexanderguzhva/9eb03ed60354d7e76cfa25e676f983ac
* Candidate `benchs/bench_ivf_fastscan.py` (sift1M): https://gist.github.com/alexanderguzhva/3cbfeba1364dd445a2bb52455966979e

mdouze should I modify `pq4_fast_scan_search_1.cpp` as well? It is somewhat cumbersome to dig through various possible sub-implementations

Pull Request resolved: facebookresearch#3276

Reviewed By: junjieqi

Differential Revision: D54943632

Pulled By: mdouze

fbshipit-source-id: 3d70066e9779039559b1734c2be99bf439058246

…ookresearch#3318)

Summary:
I noticed we have a pretty decent C API for binary indexes and please correct me if I'm wrong but we seem to be missing a couple of functions, like the ability to clone and read binary indexes. This PR provides those functions.

Pull Request resolved: facebookresearch#3318

Reviewed By: algoriddle

Differential Revision: D55469615

Pulled By: mdouze

fbshipit-source-id: 42e6f827d8b5ad6bc3efe989e47ede3aa06c1810

Summary: Pull Request resolved: facebookresearch#3328

Reviewed By: junjieqi

Differential Revision: D55476917

fbshipit-source-id: e7f64adefa07650fda32ad2300a1b933cedc9c79

Signed-off-by: Aalekh Patel <[email protected]>

Signed-off-by: Aalekh Patel <[email protected]>

Signed-off-by: Aalekh Patel <[email protected]>

Signed-off-by: Aalekh Patel <[email protected]>

…he impl.

Signed-off-by: Aalekh Patel <[email protected]>

Signed-off-by: Aalekh Patel <[email protected]>