Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/region: optimize the efficiency of random regions selecting #8205

Merged
merged 6 commits into from
May 22, 2024

Conversation

JmPotato
Copy link
Member

@JmPotato JmPotato commented May 21, 2024

What problem does this PR solve?

Issue Number: ref #7897.

What is changed and how does it work?

Optimize the efficiency of random regions selecting.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
goos: darwin
goarch: arm64
pkg: github.com/tikv/pd/pkg/core
                                                              │    old.txt    │               new.txt                │
                                                              │    sec/op     │    sec/op      vs base               │
RandomRegion/random_region_whole_range_with_size_10-8            309.9n ± 11%    290.8n ± 10%   -6.19% (p=0.041 n=6)
RandomRegion/random_regions_whole_range_with_size_10-8          2361.0n ±  1%    457.8n ±  1%  -80.61% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10-8           541.5n ±  2%    522.9n ±  1%   -3.43% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10-8          4.687µ ±  1%    1.721µ ±  1%  -63.29% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100-8           335.6n ±  1%    316.1n ±  1%   -5.81% (p=0.002 n=6)
RandomRegion/random_regions_whole_range_with_size_100-8         2579.0n ±  1%    486.3n ±  1%  -81.14% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100-8          597.4n ±  1%    655.3n ±  1%   +9.69% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100-8         5.322µ ±  1%    2.386µ ±  1%  -55.17% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000-8          396.1n ±  1%    381.7n ± 13%        ~ (p=0.065 n=6)
RandomRegion/random_regions_whole_range_with_size_1000-8        3169.0n ±  7%    711.1n ±  1%  -77.56% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000-8         715.1n ±  1%    864.9n ±  0%  +20.94% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000-8        6.581µ ±  2%    3.686µ ±  1%  -43.99% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000-8         502.2n ±  2%    518.3n ±  2%   +3.21% (p=0.002 n=6)
RandomRegion/random_regions_whole_range_with_size_10000-8        4.315µ ±  3%    1.240µ ±  8%  -71.26% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000-8        991.6n ±  1%   1288.5n ±  4%  +29.93% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000-8       9.134µ ±  2%    5.628µ ± 12%  -38.38% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100000-8        792.2n ±  3%    794.6n ±  4%        ~ (p=0.589 n=6)
RandomRegion/random_regions_whole_range_with_size_100000-8       7.198µ ±  2%    3.439µ ±  2%  -52.23% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100000-8       1.441µ ±  2%    1.968µ ±  1%  +36.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100000-8      13.49µ ±  1%    10.46µ ±  2%  -22.45% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000000-8       1.033µ ±  2%    1.044µ ±  5%        ~ (p=0.180 n=6)
RandomRegion/random_regions_whole_range_with_size_1000000-8      9.464µ ±  0%    6.068µ ±  1%  -35.89% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000000-8      1.764µ ±  3%    2.668µ ±  3%  +51.32% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000000-8     16.78µ ±  0%    14.87µ ±  2%  -11.41% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000000-8      7.340µ ±  9%    7.819µ ±  6%   +6.53% (p=0.009 n=6)
RandomRegion/random_regions_whole_range_with_size_10000000-8     74.33µ ±  4%    60.78µ ±  5%  -18.23% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000000-8     10.02µ ± 15%    12.71µ ±  9%  +26.82% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000000-8   104.13µ ±  7%    94.82µ ± 10%   -8.94% (p=0.009 n=6)
geomean                                                          2.921µ          2.092µ        -28.39%

                                                              │   old.txt    │               new.txt               │
                                                              │     B/op     │    B/op     vs base                 │
RandomRegion/random_region_whole_range_with_size_10-8             416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10-8           4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10-8            848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10-8          8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100-8            416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_100-8          4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100-8           848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100-8         8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000-8           416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_1000-8         4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000-8          848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000-8        8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000-8          416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10000-8        4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000-8         848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000-8       8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100000-8         416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_100000-8       4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100000-8        848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100000-8      8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000000-8        416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_1000000-8      4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000000-8       848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000000-8     8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000000-8       416.0 ± 0%   416.0 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10000000-8     4240.0 ± 0%   488.0 ± 0%  -88.49% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000000-8      848.0 ± 0%   448.0 ± 0%  -47.17% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000000-8    8560.0 ± 0%   584.0 ± 0%  -93.18% (p=0.002 n=6)
geomean                                                         1.847Ki        480.1       -74.62%
¹ all samples are equal

                                                              │   old.txt   │               new.txt               │
                                                              │  allocs/op  │ allocs/op   vs base                 │
RandomRegion/random_region_whole_range_with_size_10-8            4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10-8          41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10-8           7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10-8         71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100-8           4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_100-8         41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100-8          7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100-8        71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000-8          4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_1000-8        41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000-8         7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000-8       71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000-8         4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10000-8       41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000-8        7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000-8      71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_100000-8        4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_100000-8      41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_100000-8       7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_100000-8     71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_1000000-8       4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_1000000-8     41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_1000000-8      7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_1000000-8    71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
RandomRegion/random_region_whole_range_with_size_10000000-8      4.000 ± 0%   4.000 ± 0%        ~ (p=1.000 n=6) ¹
RandomRegion/random_regions_whole_range_with_size_10000000-8    41.000 ± 0%   4.000 ± 0%  -90.24% (p=0.002 n=6)
RandomRegion/random_region_given_ranges_with_size_10000000-8     7.000 ± 0%   5.000 ± 0%  -28.57% (p=0.002 n=6)
RandomRegion/random_regions_given_ranges_with_size_10000000-8   71.000 ± 0%   7.000 ± 0%  -90.14% (p=0.002 n=6)
geomean                                                          16.90        4.865       -71.21%
¹ all samples are equal

Release note

None.

@JmPotato JmPotato requested review from nolouch and rleungx May 21, 2024 08:42
Copy link
Contributor

ti-chi-bot bot commented May 21, 2024

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • HuSharp
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 21, 2024
@JmPotato JmPotato force-pushed the refine_randome_regions branch 5 times, most recently from 8764d02 to 43aa936 Compare May 22, 2024 03:16
Copy link

codecov bot commented May 22, 2024

Codecov Report

Attention: Patch coverage is 70.76923% with 19 lines in your changes are missing coverage. Please review.

Project coverage is 77.24%. Comparing base (d7c7a41) to head (12142ed).
Report is 1 commits behind head on master.

Current head 12142ed differs from pull request most recent head a932eb2

Please upload reports for the commit a932eb2 to get more accurate results.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8205      +/-   ##
==========================================
- Coverage   77.26%   77.24%   -0.02%     
==========================================
  Files         471      471              
  Lines       61425    61436      +11     
==========================================
- Hits        47459    47458       -1     
- Misses      10398    10415      +17     
+ Partials     3568     3563       -5     
Flag Coverage Δ
unittests 77.24% <70.76%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 22, 2024
pkg/core/region_tree.go Outdated Show resolved Hide resolved
pkg/core/region_tree.go Outdated Show resolved Hide resolved
pkg/core/region_tree.go Outdated Show resolved Hide resolved
// This is a fast path to reduce the unnecessary iterations.
if rangeNum == 0 {
startKey, endKey = []byte(""), []byte("")
setStartEndIndices()
if endIndex <= startIndex {
if len(endKey) > 0 && bytes.Compare(startKey, endKey) > 0 {
Copy link
Member

@HuSharp HuSharp May 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if len(endKey) > 0 && bytes.Compare(startKey, endKey) > 0 {
if len(endKey) > 0 && bytes.Compare(startKey, endKey) > 0 {

Is it possible to remove this if condition when the start and end keys are set to ""?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! Removed this check in this branch.

@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 22, 2024
@JmPotato JmPotato requested a review from HuSharp May 22, 2024 07:18
@ti-chi-bot ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels May 22, 2024
@JmPotato
Copy link
Member Author

/merge

Copy link
Contributor

ti-chi-bot bot commented May 22, 2024

@JmPotato: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Copy link
Contributor

ti-chi-bot bot commented May 22, 2024

This pull request has been accepted and is ready to merge.

Commit hash: 12142ed

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label May 22, 2024
@ti-chi-bot ti-chi-bot bot merged commit 0056569 into tikv:master May 22, 2024
15 checks passed
@JmPotato JmPotato deleted the refine_randome_regions branch May 22, 2024 07:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants