-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] fix multi collection log purge #2617
Conversation
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
26bfdee
to
501c48a
Compare
@@ -131,6 +131,7 @@ def vacuum( | |||
settings.is_persistent = True | |||
settings.persist_directory = path | |||
system = System(settings=settings) | |||
system.start() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixes a bug where if trigger_vector_segments_max_seq_id_migration()
below attempted to load a segment, it would throw because the SQLite component wasn't technically started
this scenario happens when upgrading from an old version or when any collection had not yet hit its first sync_threshold persist trigger
was not caught because the CLI test didn't add any collections, I updated the test to trigger this path
HNSWConfigurationInternal, | ||
collection.get_model() | ||
.get_configuration() | ||
.get_parameter("hnsw_configuration") | ||
.value, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using .get_model().get_configuration()
was always returning the default config
not sure if this was due to a recent change because I'm pretty sure it was working correctly when I added it
chromadb/test/db/test_log_purge.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
alternative: a new multi-collection state machine that creates records
(this is harder than just modifying the existing multi-collection state machine because invariants can't inspect bundles)
Reviewed in person with @HammadB. |
Description of changes
Fixes a bug where if any collection had dangling logs, any logs created after those by other collections would not get purged.
Also fixes a small, somewhat related bug with
chroma utils vacuum
(see PR comment below).Test plan
How are these changes tested?
Added a new regression test that was formerly failing.
Documentation Changes
Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs repository?
n/a