Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

executor: add pessimistic lock keys runtime information #19547

Merged
merged 17 commits into from
Sep 1, 2020

Conversation

crazycs520
Copy link
Contributor

What problem does this PR solve?

In pessimistic transaction, the statement maybe needs to lock keys, this PR add lock keys runtime information in executor runtime stats. Here is an example:

test> select tidb_decode_plan('8wTgMAkyOF81CTAJMAlOL0EJMAl0aW1lOjE0NS40MTA5MjVtcywgbG9vcHM6MSwgbG9ja19rZXlzOiB7BSggMzQuMDU2OTM4ASc0cmVnaW9uOjEwMjQsIGsBJhAyMDAyOA03NHJwYzozMjEuNzE4NzYxBTAccGNfY291bnQFMzR9CTI4LjQ1MzEyNSBLQgGNJAoxCTI2XzcJMAkBSGA5LjIyNTg5NTMxNjgwNglmb3IgdXBkYXRlBR4AOAm4IDUwLjUzNzgzNAFgAGwFtwQyMQFMAQQcCjIJMzFfMTBWUQA8ZGF0YTpTZWxlY3Rpb25fOS5XACAxMi42NzU5NDYuVwBELCBjb3BfdGFzazoge251bTogCfosbWF4OiA0LjI3N
   -> zc4BTFkbWluOiAyMy4xMsK1cywgYXZnOiA3Ny40NjYFECxwOTU6IDEyNS4xMTEFESh0b3RfcHJvYzogOQFwIScdXwEPJXxkIDc2LjkyNzY0NG1zfQkxNS4xOTQzMzU5Mzc1QRwzCTFfOQkxX1JCAUxsdCh0ZXN0LnQxLmEsIDQwMDAwKVEADDBucywtQEkYJE4vQQo0CTEwXzgJTzA1MjUxCXRhYmxlOnQxIe40ZXAgb3JkZXI6ZmFsc2VySAA=');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('8wTgMAkyOF81CTAJMAlOL0EJMAl0aW1lOjE0NS40MTA5MjVtcywgbG9vcHM6MSwgbG9ja19rZXlzOiB7BSggMzQuMDU2OTM4ASc0cmVnaW9uOjEwMjQsIGsBJhAyMDAyOA03NHJwYzozMjEuNzE4NzYxBTAccGNfY291bnQFMzR9CTI4LjQ1MzEyNSBLQgGNJAoxCTI2XzcJMAkBSGA5LjIyNTg5NTMxNjgwNglmb3IgdX                                           |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     id                     task         estRows               operator info                 actRows    execution info                                                                                                                                                             memory              disk |
|     Update_5               root         0                     N/A                           0          time:145.410925ms, loops:1, lock_keys: {time:34.056938ms, region:1024, keys:20028, lock_rpc:321.718761ms, rpc_count:1024}                                                  28.453125 KB        N/A  |
|     └─SelectLock_7         root         20029.225895316806    for update                    20028      time:50.537834ms, loops:21                                                                                                                                                 N/A                 N/A  |
|       └─TableReader_10     root         20029.225895316806    data:Selection_9              20028      time:12.675946ms, loops:21, cop_task: {num: 1024, max: 4.277786ms, min: 23.12µs, avg: 77.466µs, p95: 125.111µs, tot_proc: 9ms, rpc_num: 1024, rpc_time: 76.927644ms}    15.1943359375 KB    N/A     |
|         └─Selection_9      cop[tikv]    20029.225895316806    lt(test.t1.a, 40000)          0          time:0ns, loops:0                                                                                                                                                          N/A                 N/A  |
|           └─TableScan_8    cop[tikv]    25251                 table:t1, keep order:false    0          time:0ns, loops:0                                                                                                                                                          N/A                 N/A  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
test> select tidb_decode_plan('4ATwdTAJNl85CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzQJMQl0aW1lOjUwLjgxNzg0MW1zLCBsb29wczoyLCBQYXJ0aWFsQ29uY3VycmVuY3k6NSwgRmluYWxDb25jdXJyZW5jeTo1LCBsb2NrX2tleXM6IHsFUSAyNS42OTc4MzYBUVhyZWdpb246MTAyNCwga2V5czoyMDAyOAFqQGNrX3JwYzoxNTQuNDM5NTA5BTAIcGNfBawFM2x9CTE4OS42ODc1IEtCCU4vQQoxCTI2XzEyCTAJAUhgOS4yMjU4OTUzMTY4MDYJZm9yIHVwZGF0ZQUeBDgJBZAYNTAuNzI0OTLhAAAxAU0BBBwKMgkzMl8xNFZRABhpbmRle
   -> DpJAQYsUmFuZ2VTY2FuXzEzLl4AHDE4LjcxNzc3Be4EbG8lP4gxLCBjb3BfdGFzazoge251bTogMSwgbWF4OjExLjE4NDU0NQHsDHByb2MtOggwLCAB+hErAQwFvAAgBTFMMTk4NW1zfQkyODEuMzA2NjQwNjI1EgwzCTEzAZQEMV9SFAEkdGFibGU6dDEsIAnNGGEoYSksIHIBzjA6Wy1pbmYsNDAwMDApIZ48ZXAgb3JkZXI6ZmFsc2UJMClAEDBucywgCdokMAlOL0EJTi9BCg==');
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_decode_plan('4ATwdTAJNl85CTAJMQlmdW5jczpjb3VudCgxKS0+Q29sdW1uIzQJMQl0aW1lOjUwLjgxNzg0MW1zLCBsb29wczoyLCBQYXJ0aWFsQ29uY3VycmVuY3k6NSwgRmluYWxDb25jdXJyZW5jeTo1LCBsb2NrX2tleXM6IHsFUSAyNS42OTc4MzYBUVhyZWdpb246MTAyNCwga2V5czoyMDAyOAFqQGNrX3JwYzoxNTQuNDM5NT                                                                     |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|     id                    task         estRows               operator info                                                 actRows    execution info                                                                                                                                                        memory              disk |
|     HashAgg_9             root         1                     funcs:count(1)->Column#4                                      1          time:50.817841ms, loops:2, PartialConcurrency:5, FinalConcurrency:5, lock_keys: {time:25.697836ms, region:1024, keys:20028, lock_rpc:154.439509ms, rpc_count:1024}    189.6875 KB         N/A  |
|     └─SelectLock_12       root         20029.225895316806    for update                                                    20028      time:50.724941ms, loops:21                                                                                                                                            N/A                 N/A  |
|       └─IndexReader_14    root         20029.225895316806    index:IndexRangeScan_13                                       20028      time:18.717776ms, loops:21, cop_task: {num: 1, max:11.184545ms, proc_keys: 0, rpc_num: 1, rpc_time: 11.181985ms}                                                      281.306640625 KB    N/A  |
|         └─IndexScan_13    cop[tikv]    20029.225895316806    table:t1, index:a(a), range:[-inf,40000), keep order:false    0          time:0ns, loops:0                                                                                                                                                     N/A                 N/A  |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

What is changed and how it works?

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Manual test

Side effects

  • Performance regression
    • Consumes more CPU
    • Consumes more MEM

Release note

  • Add pessimistic lock keys runtime information

Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
Signed-off-by: crazycs520 <[email protected]>
@crazycs520 crazycs520 requested a review from a team as a code owner August 27, 2020 12:53
@crazycs520 crazycs520 requested review from wshwsh12 and removed request for a team August 27, 2020 12:53
@github-actions github-actions bot added the sig/execution SIG execution label Aug 27, 2020
executor/executor.go Show resolved Hide resolved
store/tikv/pessimistic.go Show resolved Hide resolved
util/execdetails/execdetails.go Show resolved Hide resolved
Copy link
Member

@wjhuang2016 wjhuang2016 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 1, 2020
Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Sep 1, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 1, 2020
@qw4990
Copy link
Contributor

qw4990 commented Sep 1, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 1, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot
Copy link
Contributor

@crazycs520 merge failed.

@zz-jason
Copy link
Member

zz-jason commented Sep 1, 2020

/merge

@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit 915d84d into pingcap:master Sep 1, 2020
ti-srebot pushed a commit to ti-srebot/tidb that referenced this pull request Sep 24, 2020
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #20199

crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Oct 9, 2020
crazycs520 added a commit to ti-srebot/tidb that referenced this pull request Oct 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/executor sig/execution SIG execution status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants