Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "*: make unified-pool use FuturePool (#15925)" #16050

Merged
merged 2 commits into from
Dec 21, 2023

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Nov 22, 2023

This reverts commit 7be1b17.

What is changed and how it works?

Issue Number: Close #16015

What's Changed:

Revert "*: make unified-pool use FuturePool (#15925)"
- revert due to performance regression

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM
  • Breaking backward compatibility

Release note

None

Copy link
Contributor

ti-chi-bot bot commented Nov 22, 2023

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • bufferflies
  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 22, 2023
@nolouch
Copy link
Contributor Author

nolouch commented Nov 22, 2023

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 22, 2023
@nolouch
Copy link
Contributor Author

nolouch commented Nov 23, 2023

/release

Copy link
Contributor

ti-chi-bot bot commented Dec 8, 2023

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 8, 2023
@nolouch nolouch closed this Dec 8, 2023
@nolouch nolouch reopened this Dec 20, 2023
@nolouch nolouch force-pushed the revert branch 3 times, most recently from 55fa2a5 to a614eb6 Compare December 20, 2023 09:14
@nolouch
Copy link
Contributor Author

nolouch commented Dec 20, 2023

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Dec 20, 2023
@nolouch nolouch removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 20, 2023
@nolouch
Copy link
Contributor Author

nolouch commented Dec 20, 2023

Benchmark with sysbench oltp_point_select in 3 high performance machines:

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                72
On-line CPU(s) list:   0-71
Thread(s) per core:    2
Core(s) per socket:    18
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Gold 6240 CPU @ 2.60GHz
Stepping:              7
CPU MHz:               3900.000
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000
BogoMIPS:              5200.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              25344K
NUMA node0 CPU(s):     0-17,36-53
NUMA node1 CPU(s):     18-35,54-71
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts pku ospke avx512_vnni md_clear flush_l1d arch_capabilities

TiKV: 3* 16c 48GB Memory

Command:

sysbench oltp_point_select run --mysql-host=tidb-dsn.perftest-component-check-001-tps-5251026-1-128 --mysql-port=4000 --mysql-user=root --db-driver=mysql --mysql-db=test --threads=100 --time=600 --report-interval=10 --tables=32 --table-size=10000000 --mysql-ignore-errors=1062,2013,8028,9002,9007 --auto-inc=false

Base

hash 1819bb8f077aba4f4a28eb0aa89d43fcd3f873d7:

result:

SQL statistics:                                                                                                                                 │··········································································································································
    queries performed:                                                                                                                          │··········································································································································
        read:                            128135121                                                                                              │··········································································································································
        write:                           0                                                                                                      │··········································································································································
        other:                           0                                                                                                      │··········································································································································
        total:                           128135121                                                                                              │··········································································································································
    transactions:                        128135121 (213554.33 per sec.)                                                                         │··········································································································································
    queries:                             128135121 (213554.33 per sec.)                                                                         │··········································································································································
    ignored errors:                      0      (0.00 per sec.)                                                                                 │··········································································································································
    reconnects:                          0      (0.00 per sec.)                                                                                 │··········································································································································
                                                                                                                                                │··········································································································································
General statistics:                                                                                                                             │··········································································································································
    total time:                          600.0105s                                                                                              │··········································································································································
    total number of events:              128135121                                                                                              │··········································································································································
                                                                                                                                                │··········································································································································
Latency (ms):                                                                                                                                   │··········································································································································
         min:                                    0.18                                                                                           │··········································································································································
         avg:                                    0.47                                                                                           │··········································································································································
         max:                                  214.99                                                                                           │··········································································································································
         95th percentile:                        0.65                                                                                           │··········································································································································
         sum:                             59954380.22                                                                                           │··········································································································································
                                                                                                                                                │··········································································································································
Threads fairness:                                                                                                                               │··········································································································································
    events (avg/stddev):           1281351.2100/52471.59                                                                                        │··········································································································································
    execution time (avg/stddev):   599.5438/0.02                                                                                                │··········································································································································
                                                 

Bad PR :

hash: 7be1b17d72870bb95fcb7443b1f945f9c59657fa

SQL statistics:                                                                                                                                 │··········································································································································
    queries performed:                                                                                                                          │··········································································································································
        read:                            125242650                                                                                              │··········································································································································
        write:                           0                                                                                                      │··········································································································································
        other:                           0                                                                                                      │··········································································································································
        total:                           125242650                                                                                              │··········································································································································
    transactions:                        125242650 (208733.50 per sec.)                                                                         │··········································································································································
    queries:                             125242650 (208733.50 per sec.)                                                                         │··········································································································································
    ignored errors:                      0      (0.00 per sec.)                                                                                 │··········································································································································
    reconnects:                          0      (0.00 per sec.)                                                                                 │··········································································································································
                                                                                                                                                │··········································································································································
General statistics:                                                                                                                             │··········································································································································
    total time:                          600.0109s                                                                                              │··········································································································································
    total number of events:              125242650                                                                                              │··········································································································································
                                                                                                                                                │··········································································································································
Latency (ms):                                                                                                                                   │··········································································································································
         min:                                    0.17                                                                                           │··········································································································································
         avg:                                    0.48                                                                                           │··········································································································································
         max:                                  231.83                                                                                           │··········································································································································
         95th percentile:                        0.69                                                                                           │··········································································································································
         sum:                             59956408.02                                                                                           │··········································································································································
                                                                                                                                                │··········································································································································
Threads fairness:                                                                                                                               │··········································································································································
    events (avg/stddev):           1252426.5000/76790.31                                                                                        │··········································································································································
    execution time (avg/stddev):   599.5641/0.03                                                                                                │··········································································································································
                                                   

This PR:

result

SQL statistics:                                                                                                                                 │··········································································································································
    queries performed:                                                                                                                          │··········································································································································
        read:                            128335038                                                                                              │··········································································································································
        write:                           0                                                                                                      │··········································································································································
        other:                           0                                                                                                      │··········································································································································
        total:                           128335038                                                                                              │··········································································································································
    transactions:                        128335038 (213887.42 per sec.)                                                                         │··········································································································································
    queries:                             128335038 (213887.42 per sec.)                                                                         │··········································································································································
    ignored errors:                      0      (0.00 per sec.)                                                                                 │··········································································································································
    reconnects:                          0      (0.00 per sec.)                                                                                 │··········································································································································
                                                                                                                                                │··········································································································································
General statistics:                                                                                                                             │··········································································································································
    total time:                          600.0108s                                                                                              │··········································································································································
    total number of events:              128335038                                                                                              │··········································································································································
                                                                                                                                                │··········································································································································
Latency (ms):                                                                                                                                   │··········································································································································
         min:                                    0.17                                                                                           │··········································································································································
         avg:                                    0.47                                                                                           │··········································································································································
         max:                                  305.91                                                                                           │··········································································································································
         95th percentile:                        0.64                                                                                           │··········································································································································
         sum:                             59956242.92                                                                                           │··········································································································································
                                                                                                                                                │··········································································································································
Threads fairness:                                                                                                                               │··········································································································································
    events (avg/stddev):           1283350.3800/16773.28                                                                                        │··········································································································································
    execution time (avg/stddev):   599.5624/0.02 

@nolouch
Copy link
Contributor Author

nolouch commented Dec 21, 2023

PTAL @glorv @bufferflies

@@ -754,6 +791,12 @@ mod metrics {
&["name"]
)
.unwrap();
pub static ref UNIFIED_READ_POOL_RUNNING_THREADS: IntGaugeVec = register_int_gauge_vec!(
"tikv_unified_read_pool_thread_count",
"The number of running threads in the unified read pool",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"The number of running threads in the unified read pool",
"The number of running tasks in the unified read pool",

maybe pending task is more readable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its threads in here.

@ti-chi-bot ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 21, 2023
@ti-chi-bot ti-chi-bot bot removed the status/LGT1 Indicates that a PR has LGTM 1. label Dec 21, 2023
@ti-chi-bot ti-chi-bot bot added the status/LGT2 Indicates that a PR has LGTM 2. label Dec 21, 2023
@bufferflies
Copy link
Contributor

/merge

Copy link
Contributor

ti-chi-bot bot commented Dec 21, 2023

@bufferflies: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

Copy link
Contributor

ti-chi-bot bot commented Dec 21, 2023

This pull request has been accepted and is ready to merge.

Commit hash: 1642e33

@ti-chi-bot ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 21, 2023
Copy link
Contributor

ti-chi-bot bot commented Dec 21, 2023

@nolouch: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot bot merged commit a0e8a7a into tikv:master Dec 21, 2023
7 checks passed
@ti-chi-bot ti-chi-bot bot added this to the Pool milestone Dec 21, 2023
@nolouch nolouch deleted the revert branch December 21, 2023 09:22
nolouch added a commit to ti-chi-bot/tikv that referenced this pull request Dec 21, 2023
close tikv#16015

Revert "*: make unified-pool use FuturePool (tikv#15925)"
- revert due to performance regression

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
nolouch added a commit to ti-chi-bot/tikv that referenced this pull request Dec 21, 2023
close tikv#16015

Revert "*: make unified-pool use FuturePool (tikv#15925)"
- revert due to performance regression

Signed-off-by: nolouch <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
Signed-off-by: nolouch <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
4 participants