[ENH] Add rust rendezvous hashing and errors

[HammadB]

Description of changes

Summarize the changes made by this PR.

• Improvements & Bug fixes
  • /
• New functionality
  • This adds the rendezvous hashing to the rust code so that it may be used by assigners.
  • It also introduces our philosophy on error handling in the rust codebase. We will use the 17 standard error codes in the grpc codebase, this is used so lower levels can send their codes higher, and in higher levels of the code we can Box<> a dyn Chroma Error to allow for generic handling over the codes. Lower levels will have to be specific, at the expense of verbosity in the errors they propagate and wrap. We will use the thiserror library, widely adopted in rust, in order to easily derive our errors. If a type needs to wrap a source error, thiserror provides #[from] in order to allow callers to have more nuanced information. After much research, my philosophy was informed by this post - https://mmapped.blog/posts/12-rust-error-handling.html.

Test plan

How are these changes tested?

• Tests pass locally with cargo test

Documentation Changes

None required. Please evaluate the documentation quality of the code itself.

[HammadB]

Current dependencies on/for this PR:

• main
  • PR [CHORE] Initialize rust #1499
    • PR [ENH] Add Rust Configuration #1500
      • PR [CHORE] Add Rust CI & Basic Dockerfile #1504
        • PR [ENH] Add rust rendezvous hashing and errors #1508 👈
          • PR [ENH] Add rust assignmenment policy and config management #1510
            • PR [ENH] Add rust protobufs and conversion. Add build.rs, protobufs, and conversions #1513
              • PR [ENH] Add rust hnswlib bindings, index interface #1516
                • PR [ENH] Add component system + memberlist provider #1519
                  • PR [ENH] add Rust pulsar and topic management #1528
                  • PR [ENH] Rust SysDB #1529
                  • PR [ENH] Add rust message id conversion for pulsar #1531
                  • PR [ENH] Refactor system to allow mutable handles, wrapped messages, single M recievers #1539
                  • PR [ENH] Add ingest dispatch + scheduler + segment #1542
                  • PR [ENH] Add rust grpc server #1548

This stack of pull requests is managed by Graphite.

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readbility, Modularity, Intuitiveness)

[HammadB]

This is an example of how to create errors.

[HammadB]

We must always do this, it's not compile time enforced, I can't find a good way to do that right now, but higher levels of code can Box<dyn ChromaError> this way and rely on codes.

[beggers]

Looks good! One small comment nit but that's it :)

[beggers]

Note to myself: we need this because count() on a rust Iterator consumes the iterator and size_hint() explicitly states that iterators don't have to implement it correctly. So there's no way to check the size of members at the top of the function.