kafka + sql storage variant, reusing streams variant datamodel #1012
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
November 19, 2020 14:59
|
Thanks! |
|
I think the next evolution of this implementation is to add upsert-style functionality to the SQL storage layer so that we can use log compaction. Once that is done I think this implementation will be very good. |
EricWittmann
added a commit
that referenced
this pull request
Nov 24, 2020
…utils/data model (#1026) * Added a Kafka+SQL storage variant * introduced overlays for application.properties to avoid putting all properties in one file * Fixed the ksql tests - they all pass! * added some logging to the merge properties mojo * push the UUID into the payload and make the kafka message key the artifactId to ensure ordering * fix selectArtifactMetaDataByGlobalId query bug and add reproducer test * some tweaks based on perf testing * kafka + sql storage variant, reusing streams variant datamodel (#1012) * kafka + sql storage variant, reusing streams variant datamodel * ksql - integration tests * fix streams storage * fixed some bugs in the ksql modified impl, and modified some tests to handle new async behavior * minor TODO * run integration tests and fix storage bug (#1028) * fix ksql storage - error create/update artifact with metadata (#1029) * update after some PR feedback * remove some debug methods * updated the perftest readme Co-authored-by: Fabian Martinez <[email protected]> Co-authored-by: Fabian Martinez <[email protected]>
carlesarnal
pushed a commit
that referenced
this pull request
Nov 25, 2020
…utils/data model (#1026) * Added a Kafka+SQL storage variant * introduced overlays for application.properties to avoid putting all properties in one file * Fixed the ksql tests - they all pass! * added some logging to the merge properties mojo * push the UUID into the payload and make the kafka message key the artifactId to ensure ordering * fix selectArtifactMetaDataByGlobalId query bug and add reproducer test * some tweaks based on perf testing * kafka + sql storage variant, reusing streams variant datamodel (#1012) * kafka + sql storage variant, reusing streams variant datamodel * ksql - integration tests * fix streams storage * fixed some bugs in the ksql modified impl, and modified some tests to handle new async behavior * minor TODO * run integration tests and fix storage bug (#1028) * fix ksql storage - error create/update artifact with metadata (#1029) * update after some PR feedback * remove some debug methods * updated the perftest readme Co-authored-by: Fabian Martinez <[email protected]> Co-authored-by: Fabian Martinez <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Same as #1006