Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Not for merging and please ignore. #8996

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

gotjosh
Copy link
Contributor

@gotjosh gotjosh commented Aug 14, 2024

What this PR does

N/A only here for illustrative purposes

Which issue(s) this PR fixes or relates to

N/A

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@gotjosh gotjosh force-pushed the dimitar/ingester/consume-latency-push-sharding branch from 05eae9e to 0d4a616 Compare August 29, 2024 09:19
@dimitarvdimitrov dimitarvdimitrov force-pushed the dimitar/ingester/consume-latency-push-sharding branch from 16f8a12 to e28dbab Compare September 19, 2024 17:34
dimitarvdimitrov and others added 6 commits September 20, 2024 15:12
Signed-off-by: Dimitar Dimitrov <[email protected]>
Signed-off-by: Dimitar Dimitrov <[email protected]>
Signed-off-by: Dimitar Dimitrov <[email protected]>
Signed-off-by: Dimitar Dimitrov <[email protected]>
* kafka replay speed: adjust batchingQueueCapacity

I made 2000 up when we were flushing individual series to the channel.
Then 2000 might have made sense, but when flushing whole WriteRequests
a capacity of 1 should be sufficient.

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Increase errCh capacity

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Explain why +1

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Set capacity to 5

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Update pkg/storage/ingest/pusher.go

Co-authored-by: gotjosh <[email protected]>

* Improve test

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Update pkg/storage/ingest/pusher.go

---------

Signed-off-by: Dimitar Dimitrov <[email protected]>
Co-authored-by: gotjosh <[email protected]>
* kafka replay speed: rename CLI flags

Make them a bit more consistent on what they mean and add better descriptions.

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Clarify metrics

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Rename flags

Co-authored-by: gotjosh <[email protected]>

* Update docs

Signed-off-by: Dimitar Dimitrov <[email protected]>

---------

Signed-off-by: Dimitar Dimitrov <[email protected]>
Co-authored-by: gotjosh <[email protected]>
@dimitarvdimitrov dimitarvdimitrov force-pushed the dimitar/ingester/consume-latency-push-sharding branch from c0d740c to fb30f39 Compare September 20, 2024 13:12
* kafka replay speed: add support for metadata & source

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Remove completed TODO

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Use a single map

Signed-off-by: Dimitar Dimitrov <[email protected]>

* Make tests compile again

Signed-off-by: Dimitar Dimitrov <[email protected]>

---------

Signed-off-by: Dimitar Dimitrov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants