-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Check bucket series time range when decoding chunks from index #3367
Conversation
Running benchmarks is tricky but there are ways to make it reliable and more trustworthy.
|
Anyway good work I think it makes sense, wonder if we can clean decodeWithReq a bit more 🤗 Some reference for benchmarks: https://gist.github.com/bwplotka/3b853c31ed11e77c975b9df45d105d74 |
I finished the benchmarks and the new results between this pr and the current master are below. Seems it is not very good when there are not many samples per series.
|
After one more optimization, I think the performance is good now. That's the result:
|
c0b6a99
to
2f0ee7d
Compare
Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Ben Ye <[email protected]>
2f0ee7d
to
79f0eff
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ref0 += d.Varint64() | ||
t0 = maxt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be done before if maxt < req.MinTime
, otherwise the next for
iteration will read invalid data. This PR #3440 fixes it.
…s-io#3367) * check bucket series time range when decoding chunks from index Signed-off-by: Ben Ye <[email protected]> * reduce alloc Signed-off-by: Ben Ye <[email protected]> Signed-off-by: Oghenebrume50 <[email protected]>
Signed-off-by: Ben Ye [email protected]
Changes
This idea behind this pr is to check the chunk time range when decoding the series from the index. This change can benefit a lot the
skipChunk
queries because it can immediately return if it finds a valid chunk instead of decoding all the chunks in this series.Verification
Uses the benchmarks from #3366. Benchmark and compare between this pr and current master.
SkipChunk
query.It is expected that this type of query should be much faster than before.
The expected performance of this should be almost the same or a little bit slower than before. However, this pr is about 10% - 20% slower when there are a lot of samples in one series. Not sure how to improve this case. I think there is no extra overhead except we are using more if-else statements than before when decoding the index.