Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

store: Optimized common cases for time selecting smaller amount of series. Avoid looking up symbols. #3531

Merged
merged 3 commits into from
Dec 4, 2020
Merged

store: Optimized common cases for time selecting smaller amount of series. Avoid looking up symbols. #3531

merged 3 commits into from
Dec 4, 2020

Commits on Dec 2, 2020

  1. store: Optimized common cases for time selecting tiny amount series. … 

    …Avoid looking up symbols.

Key part: Majority of query latency comes from symbols.Lookup.

Also cleaned up the code.

## Benchmarks

Baseline:
```
/tmp/___BenchmarkTelemeterRealData_Series_in_github_com_thanos_io_thanos_pkg_store -test.v -test.bench ^\QBenchmarkTelemeterRealData_Series\E$ -test.run ^$ -test.benchtime=1m -test.benchmem
goos: linux
goarch: amd64
pkg: github.com/thanos-io/thanos/pkg/store
BenchmarkTelemeterRealData_Series
Built index header; Starting
BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E
BenchmarkTelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12         	      43	1866550478 ns/op	395648807 B/op	 4348808 allocs/op
BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E
BenchmarkTelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12        	      50	1417560470 ns/op	160595177 B/op	 3017177 allocs/op
BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E
BenchmarkTelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12      	     392	 168130862 ns/op	49921099 B/op	  493076 allocs/op
BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E
BenchmarkTelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12           	      14	5295236741 ns/op	1083815895 B/op	11798226 allocs/op
BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E
BenchmarkTelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12          	      18	4149971387 ns/op	436568360 B/op	 8577264 allocs/op
PASS

Process finished with exit code 0
```

New
```
benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt
name                                                                old time/op    new time/op    delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        1.87s ± 0%     1.66s ± 0%  -10.85%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       1.42s ± 0%     0.25s ± 0%  -82.17%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12     168ms ± 0%     157ms ± 0%   -6.72%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12          5.30s ± 0%     4.53s ± 0%  -14.53%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         4.15s ± 0%     0.68s ± 0%  -83.67%

name                                                                old alloc/op   new alloc/op   delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        396MB ± 0%     396MB ± 0%   -0.00%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       161MB ± 0%      95MB ± 0%  -40.95%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12    49.9MB ± 0%    49.9MB ± 0%   +0.00%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12         1.08GB ± 0%    1.08GB ± 0%   -0.00%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         437MB ± 0%     228MB ± 0%  -47.77%

name                                                                old allocs/op  new allocs/op  delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        4.35M ± 0%     4.35M ± 0%   -0.00%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       3.02M ± 0%     0.05M ± 0%  -98.36%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12      493k ± 0%      493k ± 0%   +0.00%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12          11.8M ± 0%     11.8M ± 0%   -0.00%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         8.58M ± 0%     0.02M ± 0%  -99.78%
```
Cmp:

```
benchstat -delta-test=none _dev/bench/store_symb_a.txt _dev/bench/store_symb_b.txt
name                                                                old time/op    new time/op    delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        1.87s ± 0%     1.66s ± 0%  -10.85%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       1.42s ± 0%     0.25s ± 0%  -82.17%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12     168ms ± 0%     157ms ± 0%   -6.72%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12          5.30s ± 0%     4.53s ± 0%  -14.53%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         4.15s ± 0%     0.68s ± 0%  -83.67%

name                                                                old alloc/op   new alloc/op   delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        396MB ± 0%     396MB ± 0%   -0.00%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       161MB ± 0%      95MB ± 0%  -40.95%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12    49.9MB ± 0%    49.9MB ± 0%   +0.00%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12         1.08GB ± 0%    1.08GB ± 0%   -0.00%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         437MB ± 0%     228MB ± 0%  -47.77%

name                                                                old allocs/op  new allocs/op  delta
TelemeterRealData_Series/alerts2w/01DN3SK96XDAEKRB1AN30AAW6E-12        4.35M ± 0%     4.35M ± 0%   -0.00%
TelemeterRealData_Series/alerts15s/01DN3SK96XDAEKRB1AN30AAW6E-12       3.02M ± 0%     0.05M ± 0%  -98.36%
TelemeterRealData_Series/subssyncs2w/01DN3SK96XDAEKRB1AN30AAW6E-12      493k ± 0%      493k ± 0%   +0.00%
TelemeterRealData_Series/subs2w/01DN3SK96XDAEKRB1AN30AAW6E-12          11.8M ± 0%     11.8M ± 0%   -0.00%
TelemeterRealData_Series/subs15s/01DN3SK96XDAEKRB1AN30AAW6E-12         8.58M ± 0%     0.02M ± 0%  -99.78%
```

Code: https://gist.github.com/bwplotka/cbcbbcd1802181b7785da11dcc0f5cfd


Signed-off-by: Bartlomiej Plotka <[email protected]>
    @bwplotka
    bwplotka committed Dec 2, 2020
    Configuration menu
    Copy the full SHA
    cf9b864 View commit details
    Browse the repository at this point in the history

  2. Fixed chunk mint. 

    Signed-off-by: Bartlomiej Plotka <[email protected]>
    @bwplotka
    bwplotka committed Dec 2, 2020
    Configuration menu
    Copy the full SHA
    593b0d6 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2020

  1. Fixed bug spotted by Ben! 

    Signed-off-by: Bartlomiej Plotka <[email protected]>
    @bwplotka
    bwplotka committed Dec 3, 2020
    Configuration menu
    Copy the full SHA
    97f1ea0 View commit details
    Browse the repository at this point in the history