Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

planner: Add control flag to keep or remove ORDER BY in subquery #33173

Merged
merged 2 commits into from
Apr 1, 2022

Conversation

fixdb
Copy link
Contributor

@fixdb fixdb commented Mar 16, 2022

What problem does this PR solve?

Issue Number: close #32900
Unnecessary OrderBy not removed resulting in poor performance.

Problem Summary:

What is changed and how it works?

We introduced a new control flag "tidb_remove_orderby_in_subquery" to indicate
whether to remove ORDER BY clause in subquery if the subquery doesn't have LIMIT
clause. The default value is OFF, which means no behavior change in current
system, unless users explicitly change this setting.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Add a variable "tidb_remove_orderby_in_subquery" to indicate whether
to remove ORDER BY clause in subquery if possible. Default value is OFF.

@fixdb fixdb requested a review from a team as a code owner March 16, 2022 22:24
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 16, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • morgo
  • qw4990

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Mar 16, 2022
@CLAassistant
Copy link

CLAassistant commented Mar 16, 2022

CLA assistant check
All committers have signed the CLA.

@ti-chi-bot ti-chi-bot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 16, 2022
@ti-chi-bot
Copy link
Member

Welcome @yuanhsh!

It looks like this is your first PR to pingcap/tidb 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/tidb. 😃

@sre-bot
Copy link
Contributor

sre-bot commented Mar 16, 2022

@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 16, 2022
@fixdb
Copy link
Contributor Author

fixdb commented Mar 16, 2022

@chrysan @qw4990 Please take a look.

@qw4990 qw4990 self-requested a review March 17, 2022 03:36
Copy link
Contributor

@qw4990 qw4990 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

sessionctx/variable/tidb_vars.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 17, 2022
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 17, 2022
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 18, 2022
@qw4990 qw4990 requested a review from chrysan March 18, 2022 02:35
{Scope: ScopeGlobal | ScopeSession, Name: TiDBRemoveOrderbyInSubquery, Value: BoolToOnOff(DefTiDBRemoveOrderbyInSubquery), Type: TypeBool, SetSession: func(s *SessionVars, val string) error {
s.RemoveOrderbyInSubquery = TiDBOptOn(val)
return nil
}},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not a global variable in this way. Mark the ScopeGlobal will enable the persistence to GLOBAL_VARIABLES table while the value persisted is never inherited by session (You use DefTiDBRemoveOrderbyInSubquery to init s. RemoveOrderbyInSubquery).

You can just remove ScopeGlobal or implement the full ScopeGlobal logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I have removed ScopeGlobal as suggested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looked right to me, but I'm not sure because of the force-push.

The Value: only refers to the default value, once it is persisted it will use what is stored internally. SetSession is called both when a session value is changed, and on init for new sessions (on init the val will be the global value, which matches the GLOBAL_VARIABLES -- although we have a cache of it internally called sysvar_cache).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The downside of ScopeSession only is that it doesn't persist, so each time a user wants this optimization to apply -- they'll need to SET sysvar = x. Which limits its appeal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, my fault, I mixed the situation, @yuanhsh please ignore my previous comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I will revert to previous version.
@morgo Thank you for advising!

We introduced a new contral flag "tidb_remove_orderby_in_subquery" to indicate
whether to remove ORDER BY clause in subquery if the subquery doesn't have LIMIT
clause. The default value is false, which means no behaviour change in current
system, unless users explicitly change this setting.
@fixdb
Copy link
Contributor Author

fixdb commented Mar 23, 2022

@morgo @chrysan Updated and rebased on latest master branch.

Copy link
Contributor

@chrysan chrysan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member

@chrysan: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

LGTM

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@morgo morgo self-requested a review March 23, 2022 20:27
@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 23, 2022
@qw4990
Copy link
Contributor

qw4990 commented Apr 1, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 7bc1164

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 1, 2022
@qw4990
Copy link
Contributor

qw4990 commented Apr 1, 2022

/run-mysql-test

@ti-chi-bot ti-chi-bot merged commit fa834a0 into pingcap:master Apr 1, 2022
@fixdb fixdb deleted the remove_orderby branch April 10, 2022 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unnecessary OrderBy not removed resulting in poor performance
7 participants