[ENH] Collection Configuration Storage

[atroyn]

Description of changes

This PR introduces storing collection parameters as a Configuration, a JSON-serializable defined object with specified configuration fields, adapting @HammadB's work in #1491.

Configurations are created at creation time for collections, and are immutable thereafter.

The JSON-serialized config is stored as a new text column config_json_str on the collections table.

Note that in this stack, Chroma does not actually consume the configuration for anything - this will be done in a separate stack.

This PR also updates our versions of black and mypy, which were failing with our new definitions but work with the new versions.

Test plan

Tests pass in CI except the following:

• Tests involving GRPC / Cluster. This is implemented later in this stack.
• Tests involving cross-version persist. This is implemented later in this stack, along with a user-enabled migration.
• JavaScript - @jeffchuber is taking a look here. Thanks @jeffchuber

Documentation Changes

N/A - This is not a user-facing change. Documentation around migrations are later in the stack along with the cross-version persistence tests.

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[github-actions]

Please tag your PR title with one of: [ENH | BUG | DOC | TST | BLD | PERF | TYP | CLN | CHORE]. See https://docs.trychroma.com/contributing#contributing-code-and-ideas

[atroyn]

• [ENH] Configuration UX #2437
• [ENH] Cross-Version Collection Migration #2400
• [ENH] Distributed System collection config storage #2396
• [ENH] Collection Configuration Storage #2338 👈
• [CLN] ServerAPI to use Collection model #2300 : 1 other dependent PR (#2265 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @atroyn and the rest of your teammates on Graphite

[atroyn]

Merge activity

• Jul 8, 11:00 PM EDT: @atroyn started a stack merge that includes this pull request via Graphite.
• Jul 8, 11:04 PM EDT: Graphite rebased this pull request as part of a merge.
• Jul 8, 11:43 PM EDT: @atroyn merged this pull request with Graphite.