planner: a new thread-safe lock-free plan cache #54089

qw4990 · 2024-06-18T10:06:47Z

What problem does this PR solve?

Issue Number: close #54057

Problem Summary: planner: a new thread-safe lock-free plan cache

What changed and how does it work?

I'll add more test cases in the next few PRs.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No need to test
- I checked and no code files have been changed.

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

tiprow · 2024-06-18T10:07:05Z

Hi @qw4990. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

codecov · 2024-06-18T10:25:45Z

Codecov Report

Attention: Patch coverage is 87.12121% with 17 lines in your changes missing coverage. Please review.

Project coverage is 56.1599%. Comparing base (dfa6b7c) to head (1621eb0).
Report is 52 commits behind head on master.

Additional details and impacted files

@@                Coverage Diff                @@
##             master     #54089         +/-   ##
=================================================
- Coverage   74.8567%   56.1599%   -18.6968%     
=================================================
  Files          1516       1642        +126     
  Lines        360852     615583     +254731     
=================================================
+ Hits         270122     345711      +75589     
- Misses        71179     246554     +175375     
- Partials      19551      23318       +3767

Flag	Coverage Δ
integration	`36.2468% <0.0000%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	`52.9656% <ø> (-2.2339%)`	⬇️
parser	`∅ <ø> (∅)`
br	`52.1240% <ø> (+4.1456%)`	⬆️

…an-cache-1

pkg/planner/core/plan_cache_instance.go

Rustin170506

thanks!

pkg/planner/core/plan_cache_instance.go

Co-authored-by: 二手掉包工程师 <[email protected]>

pkg/planner/core/plan_cache_instance.go

pkg/planner/core/plan_cache_instance_test.go

pkg/planner/core/plan_cache_instance.go

time-and-fate · 2024-06-24T22:44:39Z

pkg/planner/core/plan_cache_instance.go

+	if len(lastUsedTimes) < int(numToEvict) {
+		return


How could this happen?

Just for safety, to avoid index-out-of-range panic below.

Co-authored-by: Zhou Kunqin <[email protected]>

qw4990 · 2024-06-25T06:48:27Z

Thanks, all comments have been addressed @time-and-fate

time-and-fate

Others LGTM.

time-and-fate · 2024-06-25T08:23:59Z

pkg/planner/core/plan_cache_instance.go

+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package core


Need an empty line here to pass CI.

time-and-fate · 2024-06-25T08:24:09Z

pkg/planner/core/plan_cache_instance_test.go

+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+package core


time-and-fate · 2024-06-25T08:24:42Z

pkg/planner/core/plan_cache_instance.go

+	Evict(sctx sessionctx.Context) (evicted bool)
+	// MemUsage returns the total memory usage of this plan cache.
+	MemUsage(sctx sessionctx.Context) int64


Looks like we don't need sctx parameter for these two methods.

Rustin170506

Approve in advance.

Please address @time-and-fate's comment.

Rustin170506 · 2024-06-25T09:44:40Z

pkg/planner/core/plan_cache_instance.go

+	// Get gets the cached value from the cache according to key and opts.
+	Get(sctx sessionctx.Context, key string, opts any) (value any, ok bool)
+	// Put puts the key and value into the cache.
+	Put(sctx sessionctx.Context, key string, value, opts any) (succ bool)


Maybe we should add a comment to explain why we need to use them here. But I am fine if you prefer not to add it.

Added, Thanks!

ti-chi-bot · 2024-06-25T11:31:23Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hi-rustin, time-and-fate

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~pkg/planner/OWNERS~~ [hi-rustin,time-and-fate]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2024-06-25T11:31:27Z

[LGTM Timeline notifier]

Timeline:

2024-06-25 09:45:19.993634557 +0000 UTC m=+712846.479123384: ☑️ agreed by hi-rustin.
2024-06-25 11:31:25.661662869 +0000 UTC m=+719212.147151699: ☑️ agreed by time-and-fate.

fixup

f778775

ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. sig/planner SIG: Planner labels Jun 18, 2024

ti-chi-bot bot removed the do-not-merge/needs-linked-issue label Jun 18, 2024

qw4990 added 2 commits June 18, 2024 18:17

fixup

c873185

fixup

435d388

qw4990 added 3 commits June 18, 2024 18:25

make linter happy

e5e3f00

Merge remote-tracking branch 'upstream/master' into instance-level-pl…

c521185

…an-cache-1

fixup

7ad67e9

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 18, 2024

fixup

d39517f

hawkingrei reviewed Jun 18, 2024

View reviewed changes

pkg/planner/core/plan_cache_instance.go Outdated Show resolved Hide resolved

qw4990 added 2 commits June 19, 2024 09:13

fixup

c75315f

fixup

ca2636d

Rustin170506 reviewed Jun 20, 2024

View reviewed changes

pkg/planner/core/plan_cache_instance.go Show resolved Hide resolved

pkg/planner/core/plan_cache_instance.go Show resolved Hide resolved

pkg/planner/core/plan_cache_instance.go Outdated Show resolved Hide resolved

pkg/planner/core/plan_cache_instance.go Outdated Show resolved Hide resolved

qw4990 and others added 2 commits June 21, 2024 13:51

Update pkg/planner/core/plan_cache_instance.go

11824ca

Co-authored-by: 二手掉包工程师 <[email protected]>

Update pkg/planner/core/plan_cache_instance.go

a09f5ac

Co-authored-by: 二手掉包工程师 <[email protected]>

time-and-fate reviewed Jun 24, 2024

View reviewed changes

qw4990 and others added 3 commits June 25, 2024 14:21

Update pkg/planner/core/plan_cache_instance.go

bee5d7a

Co-authored-by: Zhou Kunqin <[email protected]>

Update pkg/planner/core/plan_cache_instance_test.go

8d5d054

Co-authored-by: Zhou Kunqin <[email protected]>

fixup

8be33d0

time-and-fate reviewed Jun 25, 2024

View reviewed changes

Rustin170506 approved these changes Jun 25, 2024

View reviewed changes

ti-chi-bot bot added the approved label Jun 25, 2024

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 25, 2024

qw4990 added 2 commits June 25, 2024 18:46

fixup

80a5de6

fixup

1621eb0

time-and-fate approved these changes Jun 25, 2024

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 25, 2024

ti-chi-bot bot merged commit 4869956 into pingcap:master Jun 25, 2024
20 of 23 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

planner: a new thread-safe lock-free plan cache #54089

planner: a new thread-safe lock-free plan cache #54089

qw4990 commented Jun 18, 2024 •

edited

Loading

tiprow bot commented Jun 18, 2024

codecov bot commented Jun 18, 2024 •

edited

Loading

Rustin170506 left a comment

time-and-fate Jun 24, 2024

qw4990 Jun 25, 2024

qw4990 commented Jun 25, 2024

time-and-fate left a comment

time-and-fate Jun 25, 2024

time-and-fate Jun 25, 2024

time-and-fate Jun 25, 2024

Rustin170506 left a comment

Rustin170506 Jun 25, 2024

qw4990 Jun 25, 2024

ti-chi-bot bot commented Jun 25, 2024

ti-chi-bot bot commented Jun 25, 2024

planner: a new thread-safe lock-free plan cache #54089

planner: a new thread-safe lock-free plan cache #54089

Conversation

qw4990 commented Jun 18, 2024 • edited Loading

What problem does this PR solve?

What changed and how does it work?

Check List

Release note

tiprow bot commented Jun 18, 2024

codecov bot commented Jun 18, 2024 • edited Loading

Codecov Report

Rustin170506 left a comment

Choose a reason for hiding this comment

time-and-fate Jun 24, 2024

Choose a reason for hiding this comment

qw4990 Jun 25, 2024

Choose a reason for hiding this comment

qw4990 commented Jun 25, 2024

time-and-fate left a comment

Choose a reason for hiding this comment

time-and-fate Jun 25, 2024

Choose a reason for hiding this comment

time-and-fate Jun 25, 2024

Choose a reason for hiding this comment

time-and-fate Jun 25, 2024

Choose a reason for hiding this comment

Rustin170506 left a comment

Choose a reason for hiding this comment

Rustin170506 Jun 25, 2024

Choose a reason for hiding this comment

qw4990 Jun 25, 2024

Choose a reason for hiding this comment

ti-chi-bot bot commented Jun 25, 2024

ti-chi-bot bot commented Jun 25, 2024

[LGTM Timeline notifier]

qw4990 commented Jun 18, 2024 •

edited

Loading

codecov bot commented Jun 18, 2024 •

edited

Loading