-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement capnproto replication #7659
Merged
Merged
Commits on Oct 17, 2024
-
Implement capnproto replication
Our profiles from production show that a lot of CPU and memory in receivers is used for unmarshaling protobuf messages. Although it is not possible to change the remote-write format, we have the freedom to change the protocol used for replicating timeseries data. This commit introduces a new feature in receivers where replication can be done using Cap'n Proto instead of gRPC + Protobuf. The advantage of the former protocol is that deserialization is far cheaper and fields can be accessed directly from the received message (byte slice) without allocating intermediate objects. There is an additional cost for serialization because we have to convert from Protobuf to the Cap'n proto format, but in our setup this still results in a net reduction in resource usage. Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 7a854cb - Browse repository at this point
Copy the full SHA 7a854cbView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9f53a82 - Browse repository at this point
Copy the full SHA 9f53a82View commit details -
Configuration menu - View commit details
-
Copy full SHA for 36af787 - Browse repository at this point
Copy the full SHA 36af787View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4b31096 - Browse repository at this point
Copy the full SHA 4b31096View commit details -
Configuration menu - View commit details
-
Copy full SHA for 503898d - Browse repository at this point
Copy the full SHA 503898dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 9ac8db4 - Browse repository at this point
Copy the full SHA 9ac8db4View commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ba5c325 - Browse repository at this point
Copy the full SHA ba5c325View commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cd8c0bc - Browse repository at this point
Copy the full SHA cd8c0bcView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7721d56 - Browse repository at this point
Copy the full SHA 7721d56View commit details -
Update docs/components/receive.md
Co-authored-by: Pedro Tanaka <[email protected]> Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 2057bdf - Browse repository at this point
Copy the full SHA 2057bdfView commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for a6fda55 - Browse repository at this point
Copy the full SHA a6fda55View commit details -
e2e: add receive test with capnp replication
Signed-off-by: Giedrius Statkevičius <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for acfdae2 - Browse repository at this point
Copy the full SHA acfdae2View commit details -
receive: make copy only when necessary
Signed-off-by: Giedrius Statkevičius <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 2f9878c - Browse repository at this point
Copy the full SHA 2f9878cView commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 1193690 - Browse repository at this point
Copy the full SHA 1193690View commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for bcc3f6a - Browse repository at this point
Copy the full SHA bcc3f6aView commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 186c089 - Browse repository at this point
Copy the full SHA 186c089View commit details -
Signed-off-by: Filip Petkovski <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5f908a3 - Browse repository at this point
Copy the full SHA 5f908a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 67ef38d - Browse repository at this point
Copy the full SHA 67ef38dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d337e0a - Browse repository at this point
Copy the full SHA d337e0aView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.