-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
store: Optimized common cases for time selecting smaller amount of series. Avoid looking up symbols. #3531
Merged
Commits on Dec 2, 2020
-
store: Optimized common cases for time selecting tiny amount series. …
…Avoid looking up symbols. Key part: Majority of query latency comes from symbols.Lookup. Also cleaned up the code. ## Benchmarks Baseline: ``` /tmp/___BenchmarkTelemeterRealData_Series_in_github_com_thanos_io_thanos_pkg_store -test.v -test.bench ^\QBenchmarkTelemeterRealData_Series\E$ -test.run ^$ -test.benchtime=1m -test.benchmem goos: linux goarch: amd64 pkg: github.com/thanos-io/thanos/pkg/store BenchmarkTelemeterRealData_Series Built index header; Starting BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 43 1866550478 ns/op 395648807 B/op 4348808 allocs/op BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 50 1417560470 ns/op 160595177 B/op 3017177 allocs/op BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 392 168130862 ns/op 49921099 B/op 493076 allocs/op BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 14 5295236741 ns/op 1083815895 B/op 11798226 allocs/op BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 18 4149971387 ns/op 436568360 B/op 8577264 allocs/op PASS Process finished with exit code 0 ``` New ``` benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt name old time/op new time/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.87s ± 0% 1.66s ± 0% -10.85% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 1.42s ± 0% 0.25s ± 0% -82.17% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 168ms ± 0% 157ms ± 0% -6.72% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 5.30s ± 0% 4.53s ± 0% -14.53% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 4.15s ± 0% 0.68s ± 0% -83.67% name old alloc/op new alloc/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 396MB ± 0% 396MB ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 161MB ± 0% 95MB ± 0% -40.95% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 49.9MB ± 0% 49.9MB ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.08GB ± 0% 1.08GB ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 437MB ± 0% 228MB ± 0% -47.77% name old allocs/op new allocs/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 4.35M ± 0% 4.35M ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 3.02M ± 0% 0.05M ± 0% -98.36% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 493k ± 0% 493k ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 11.8M ± 0% 11.8M ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 8.58M ± 0% 0.02M ± 0% -99.78% ``` Cmp: ``` benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt name old time/op new time/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.87s ± 0% 1.66s ± 0% -10.85% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 1.42s ± 0% 0.25s ± 0% -82.17% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 168ms ± 0% 157ms ± 0% -6.72% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 5.30s ± 0% 4.53s ± 0% -14.53% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 4.15s ± 0% 0.68s ± 0% -83.67% name old alloc/op new alloc/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 396MB ± 0% 396MB ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 161MB ± 0% 95MB ± 0% -40.95% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 49.9MB ± 0% 49.9MB ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 1.08GB ± 0% 1.08GB ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 437MB ± 0% 228MB ± 0% -47.77% name old allocs/op new allocs/op delta TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12 4.35M ± 0% 4.35M ± 0% -0.00% TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12 3.02M ± 0% 0.05M ± 0% -98.36% TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 493k ± 0% 493k ± 0% +0.00% TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12 11.8M ± 0% 11.8M ± 0% -0.00% TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12 8.58M ± 0% 0.02M ± 0% -99.78% ``` Code: https://gist.github.com/bwplotka/cbcbbcd1802181b7785da11dcc0f5cfd Signed-off-by: Bartlomiej Plotka <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cf9b864 - Browse repository at this point
Copy the full SHA cf9b864View commit details -
Signed-off-by: Bartlomiej Plotka <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 593b0d6 - Browse repository at this point
Copy the full SHA 593b0d6View commit details
Commits on Dec 3, 2020
-
Signed-off-by: Bartlomiej Plotka <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 97f1ea0 - Browse repository at this point
Copy the full SHA 97f1ea0View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.