Expose the variants of binary indexes to the C API. #2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Pull Request resolved: facebookresearch#3317 libraft packages are first published in rapidsai-nightly and moved to rapidsai after release, at which point they're removed from rapidsai-nightly In this diff we enable both channels with a preference to rapidsai (since it's before rapidsai-nightly on the command line). Reviewed By: mlomeli1 Differential Revision: D55310143 fbshipit-source-id: b85e0fda86a442f435d985ace1d7eb37209c74e1
Summary: Pull Request resolved: facebookresearch#3312 as the [#issue3258](facebookresearch#3258) mentioned, the IVFPQFastScan should have same decoding result as IVFPQ. However, current result is not as expected. In this PR/Diff, we are going to fix the decoding function Reviewed By: mdouze Differential Revision: D55264781 fbshipit-source-id: dfdae9eabceadfc5a3ebb851930d71ce3c1c654d
Summary: In the facebookresearch#3278, we to find_package to link to GTest. However, it needs to have googletest to build independently. Not everyone builds their googletest locally first. In this diff, we still try to build googletest from source and combine find_package together. Pull Request resolved: facebookresearch#3319 Test Plan: STEP 1: Install deps ``` conda install -y -q python=3.11 cmake make swig=4.0.2 mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64 ``` STEP2: Compile ``` cmake -B build \ -DBUILD_TESTING=ON \ -DBUILD_SHARED_LIBS=ON \ -DFAISS_ENABLE_GPU=OFF \ -DFAISS_ENABLE_RAFT=OFF \ -DFAISS_OPT_LEVEL=avx2 \ -DFAISS_ENABLE_C_API=ON \ -DPYTHON_EXECUTABLE=$(which python) \ -DCMAKE_BUILD_TYPE=Release \ -DBLA_VENDOR=Intel10_64_dyn \ -DCMAKE_CUDA_FLAGS="-gencode arch=compute_75,code=sm_75" \ . ``` Reviewed By: algoriddle Differential Revision: D55358059 Pulled By: junjieqi fbshipit-source-id: 95ad4a745238b88b438728de64173f99d3d50dbe
Summary: Fixes problem when compiling OpenBLAS with INTERFACE64=1 (64-bit integers). Pull Request resolved: facebookresearch#3322 Reviewed By: algoriddle Differential Revision: D55469397 Pulled By: mdouze fbshipit-source-id: 14d916fb074f6ea0f591e0324bb7b8674a624473
Summary: Currently, faiss can't build with swig version > 4.2.x. As the facebookresearch#3239 mentioned. Swig removed the support for 32bit swig/swig@9fb3a49. So SWIGTYPE_p_unsigned_long_long isn't supported any more. In this diff, we are going to remove the unsupported type from Faiss swig. Pull Request resolved: facebookresearch#3315 Test Plan: STEP 1: create a new conda env ``` conda create --name faiss_swig conda activate faiss_swig ``` STEP 2: install dependecies from conda-forge ``` conda install -y -q python=3.11 cmake make swig mkl=2023 mkl-devel=2023 numpy scipy pytest gxx_linux-64 sysroot_linux-64=2.28 -c conda-forge ``` STEP 3: CMAKE ``` cmake -B build \ -DBUILD_TESTING=ON \ -DBUILD_SHARED_LIBS=ON \ -DFAISS_ENABLE_GPU=OFF \ -DFAISS_ENABLE_RAFT=OFF \ -DFAISS_OPT_LEVEL=avx512 \ -DFAISS_ENABLE_C_API=ON \ -DPYTHON_EXECUTABLE=$(which python) \ -DCMAKE_BUILD_TYPE=Release \ -DBLA_VENDOR=Intel10_64_dyn \ -DCMAKE_CUDA_FLAGS="-gencode arch=compute_75,code=sm_75" \ . ``` STEP 4: build ``` make -C build -j faiss && make -C build -j swigfaiss ``` <img width="876" alt="Screenshot 2024-03-25 at 12 24 16 AM" src="https://github.com/facebookresearch/faiss/assets/8333898/918f0caf-398a-4361-989f-93ff547cf2b2"> Reviewed By: algoriddle Differential Revision: D55304004 Pulled By: junjieqi fbshipit-source-id: e958009dc637aa33b0e1a574a16a846a4abb1525
Summary: AVX-512 implementation for PQFastScan for QBS. For local benchmarks on 4th gen Xeon, the QPS is up to 10% higher, mostly for a single query case. But as far as I remember, production cases would show higher performance improvements. * Baseline `benchs/bench_ivf_fastscan_single_query.py` (sift1M): https://gist.github.com/alexanderguzhva/c9cde2cb5e9c7675f429623e6faa9fbf * Candidate `benchs/bench_ivf_fastscan_single_query.py` (sift1M): https://gist.github.com/alexanderguzhva/4e8530073a108f73771d38e55bc45b17 * Baseline `benchs/bench_ivf_fastscan.py` (sift1M): https://gist.github.com/alexanderguzhva/9eb03ed60354d7e76cfa25e676f983ac * Candidate `benchs/bench_ivf_fastscan.py` (sift1M): https://gist.github.com/alexanderguzhva/3cbfeba1364dd445a2bb52455966979e mdouze should I modify `pq4_fast_scan_search_1.cpp` as well? It is somewhat cumbersome to dig through various possible sub-implementations Pull Request resolved: facebookresearch#3276 Reviewed By: junjieqi Differential Revision: D54943632 Pulled By: mdouze fbshipit-source-id: 3d70066e9779039559b1734c2be99bf439058246
…ookresearch#3318) Summary: I noticed we have a pretty decent C API for binary indexes and please correct me if I'm wrong but we seem to be missing a couple of functions, like the ability to clone and read binary indexes. This PR provides those functions. Pull Request resolved: facebookresearch#3318 Reviewed By: algoriddle Differential Revision: D55469615 Pulled By: mdouze fbshipit-source-id: 42e6f827d8b5ad6bc3efe989e47ede3aa06c1810
Summary: Pull Request resolved: facebookresearch#3328 Reviewed By: junjieqi Differential Revision: D55476917 fbshipit-source-id: e7f64adefa07650fda32ad2300a1b933cedc9c79
Signed-off-by: Aalekh Patel <[email protected]>
Signed-off-by: Aalekh Patel <[email protected]>
Signed-off-by: Aalekh Patel <[email protected]>
Signed-off-by: Aalekh Patel <[email protected]>
…he impl. Signed-off-by: Aalekh Patel <[email protected]>
Signed-off-by: Aalekh Patel <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.