[ENH] Evict other versions of hnsw index of the collection when another version is fetched

[sanketkedia]

Description of changes

Summarize the changes made by this PR.
It was seen that query nodes OOM in load tests. One area where we could optimize the memory footprint is the HNSW index cache. This PR restricts the cache to keep one entry per collection. Ideally, we would like the index version of this entry to be of the latest but there is no guarantee with this PR. Once the versioning changes land, we could improve this. For e.g. one bad case could be:
For the same collection:

1. get index version v1
2. get index version v2 (> v1)
3. get index version v1 (can happen due to an inflight query that started before compaction of v2 occured) -- this will evict v2 even though it is more recent and will be used again in future

Test plan

• Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

None

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[github-actions]

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

[sanketkedia]

• [ENH] Evict other versions of hnsw index of the collection when another version is fetched #2707 👈
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sanketkedia and the rest of your teammates on Graphite

[HammadB]

I think this is fine until we have version included, and we can be refcount aware / do a deferred eviction.

Is it possible to add some basic documentation in the comments describing this behavior and the motivation?