IVF sorting routine (facebookresearch#2846)

Summary: Pull Request resolved: facebookresearch#2846 Adds a function to ivf_contrib to sort the inverted lists by size without changing the results. Also moves big_batch_search to its own module. Reviewed By: algoriddle Differential Revision: D45565880 fbshipit-source-id: 091a1c1c074f860d6953bf20d04523292fb55e1a
blevesearch · May 4, 2023 · 2d8886c · 2d8886c
1 parent 4f574d5
commit 2d8886c
Show file tree

Hide file tree

Showing 14 changed files with 613 additions and 450 deletions.
diff --git a/contrib/README.md b/contrib/README.md
@@ -69,3 +69,8 @@ Contains:
 - a Python implementation of kmeans, that can be used for special datatypes (eg. sparse matrices).
 
 - a 2-level clustering routine and a function that can apply it to train an IndexIVF
+
+### big_batch_search.py
+
+Search IVF indexes with one centroid after another. Useful for large
+databases that do not fit in RAM *and* a large number of queries.