Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bifrost] Moving Payload to be an internal bifrost type #1792

Merged
merged 3 commits into from
Aug 8, 2024
Merged

Conversation

Copy link

github-actions bot commented Aug 6, 2024

Test Results

  5 files    5 suites   7m 4s ⏱️
 41 tests  41 ✅ 0 💤 0 ❌
101 runs  101 ✅ 0 💤 0 ❌

Results for commit bd3baba.

♻️ This comment has been updated with latest results.

Copy link
Contributor

@tillrohrmann tillrohrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1 for merging.

Envelope implements a new trait `HasRecordKeys` to support future matching of records based on a `KeyMatcher`. The new appender is the canonical way to write to a single log. It tries to recycle the write buffer (although it doesn't use freelist which would be a potential future improvement) and it defers serialization to unlock the potential for better serialization write buffer management.

Appender is created by `bifrost.create_appender(log_id)?` and it's the core abstraction that `BackgroundAppender` builds upon (next PR)
Introduces a new way to enqueue messages to Bifrost. This PR adds `bifrost.create_background_append(...).start()` that starts a background appender. The background appender automatically batches appends and provides flexible control over draining it or auto-destruction when all senders are dropped (see `detach()` on `AppenderHandle`).

With this, `ActionEffectHandler` makes use of this new ability and will not block the partition processor loop when writing effects to bifrost.
Hiding the details of Payload. This type will take a different shape in the future.
@AhmedSoliman AhmedSoliman merged commit bd3baba into main Aug 8, 2024
19 checks passed
@AhmedSoliman AhmedSoliman deleted the pr1792 branch August 8, 2024 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants