Planner: planner without `DAGQueryBlock` #5381

SeaRise · 2022-07-15T07:26:04Z

What problem does this PR solve?

Issue Number: ref #4739

Problem Summary:

What is changed and how it works?

remove PhysicalSource
implement PlanQuerySource
Planner run without DAGQueryBlock

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

None

) ref pingcap#4739

ti-chi-bot · 2022-07-15T07:26:05Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

windtalker
ywqzzy

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

dbms/src/Flash/Planner/PlanQuerySource.cpp

…emove_qb

SeaRise · 2022-07-25T02:58:56Z

/run-integration-test

sre-bot · 2022-08-01T03:44:09Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18991      9485             50.06%    216933  95780        55.85%

full coverage report (for internal network access only)

dbms/src/Flash/Planner/PhysicalPlanVisitor.h

dbms/src/Flash/Planner/plans/PhysicalWindow.cpp

SeaRise · 2022-08-01T15:03:38Z

/run-all-tests

sre-bot · 2022-08-01T15:41:17Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18991      9480             50.08%    216933  95754        55.86%

full coverage report (for internal network access only)

dbms/src/Flash/Planner/plans/PhysicalAggregation.cpp

dbms/src/Flash/Planner/plans/PhysicalJoin.cpp

windtalker

LGTM

ywqzzy

LGTM

ywqzzy · 2022-08-02T09:04:49Z

dbms/src/Flash/Planner/PhysicalPlan.cpp

+    auto & list_based_executors_order = dag_context.list_based_executors_order;
+    PhysicalPlanVisitor::visitPostOrder(root_node, [&](const PhysicalPlanNodePtr & plan) {
+        assert(plan);
+        if (plan->isRecordProfileStreams())


But FinalProjection.isRecordProfileStreams() == false, so I think it's better not to add likely

SeaRise · 2022-08-02T09:29:06Z

/merge

ti-chi-bot · 2022-08-02T09:29:07Z

@SeaRise: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-08-02T09:29:09Z

This pull request has been accepted and is ready to merge.

Commit hash: 3140faa

SeaRise · 2022-08-02T09:42:48Z

/run-integration-test

SeaRise · 2022-08-02T10:11:27Z

/merge

ti-chi-bot · 2022-08-02T10:11:28Z

@SeaRise: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2022-08-02T10:11:30Z

This pull request has been accepted and is ready to merge.

Commit hash: 00409a5

sre-bot · 2022-08-02T10:45:11Z

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18991      9483             50.07%    216933  95783        55.85%

full coverage report (for internal network access only)

SeaRise added 5 commits July 12, 2022 17:01

Feature Branch: merge master to planner_refactory branch (pingcap#5353

05fd9bd

) ref pingcap#4739

u

0d7ef30

update

09a5e03

udpate

e5b8cda

y

1bda85a

ti-chi-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 15, 2022

SeaRise mentioned this pull request Jul 15, 2022

interpreter planner refactor #4739

Closed

13 tasks

fix

c53dcff

SeaRise force-pushed the planner_refactory branch from 05fd9bd to 7693e14 Compare July 18, 2022 10:14

SeaRise added 2 commits July 19, 2022 00:23

merge

8312b95

update

0676e16

SeaRise commented Jul 19, 2022

View reviewed changes

dbms/src/Flash/Planner/PlanQuerySource.cpp Outdated Show resolved Hide resolved

SeaRise added 2 commits July 20, 2022 11:53

update

353a68b

update

e97bb7b

ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 20, 2022

SeaRise added 4 commits July 20, 2022 12:10

Update dbms/src/Flash/Planner/PlanQuerySource.cpp

f0f6d86

u

a9037af

Merge branch 'remove_qb' of https://github.com/SeaRise/tiflash into r…

8046f68

…emove_qb

add disable planner test

65a7e3e

SeaRise added 6 commits July 25, 2022 14:38

u

69d672f

Merge branch 'planner_refactory' into remove_qb

1b6064c

merge planner_refactory

b5ff7cf

fix

03a0ab5

Merge branch 'planner_refactory' into remove_qb

4a25811

wait for tests fixed

6e2dcc7

windtalker reviewed Aug 1, 2022

View reviewed changes

dbms/src/Flash/Planner/PhysicalPlanVisitor.h Outdated Show resolved Hide resolved

SeaRise commented Aug 1, 2022

View reviewed changes

dbms/src/Flash/Planner/plans/PhysicalWindow.cpp Show resolved Hide resolved

address comments

23937bd

SeaRise requested a review from windtalker August 2, 2022 03:08

SeaRise commented Aug 2, 2022

View reviewed changes

dbms/src/Flash/Planner/plans/PhysicalAggregation.cpp Outdated Show resolved Hide resolved

SeaRise commented Aug 2, 2022

View reviewed changes

dbms/src/Flash/Planner/plans/PhysicalJoin.cpp Outdated Show resolved Hide resolved

windtalker approved these changes Aug 2, 2022

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Aug 2, 2022

SeaRise added 2 commits August 2, 2022 15:22

address comments

0a4b465

update

3140faa

ywqzzy approved these changes Aug 2, 2022

View reviewed changes

ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Aug 2, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 2, 2022

fix test

00409a5

ti-chi-bot removed the status/can-merge Indicates a PR has been approved by a committer. label Aug 2, 2022

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Aug 2, 2022

ti-chi-bot merged commit 0f26fe8 into pingcap:planner_refactory Aug 2, 2022

SeaRise deleted the remove_qb branch December 13, 2022 10:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Planner: planner without `DAGQueryBlock` #5381

Planner: planner without `DAGQueryBlock` #5381

SeaRise commented Jul 15, 2022 •

edited

Loading

ti-chi-bot commented Jul 15, 2022 •

edited

Loading

SeaRise commented Jul 25, 2022

sre-bot commented Aug 1, 2022

SeaRise commented Aug 1, 2022

sre-bot commented Aug 1, 2022

windtalker left a comment

ywqzzy left a comment

ywqzzy Aug 2, 2022

SeaRise Aug 2, 2022

SeaRise commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

SeaRise commented Aug 2, 2022

SeaRise commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

sre-bot commented Aug 2, 2022

Planner: planner without DAGQueryBlock #5381

Planner: planner without DAGQueryBlock #5381

Conversation

SeaRise commented Jul 15, 2022 • edited Loading

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Jul 15, 2022 • edited Loading

SeaRise commented Jul 25, 2022

sre-bot commented Aug 1, 2022

Coverage for changed files

Coverage summary

SeaRise commented Aug 1, 2022

sre-bot commented Aug 1, 2022

Coverage for changed files

Coverage summary

windtalker left a comment

Choose a reason for hiding this comment

ywqzzy left a comment

Choose a reason for hiding this comment

ywqzzy Aug 2, 2022

Choose a reason for hiding this comment

SeaRise Aug 2, 2022

Choose a reason for hiding this comment

SeaRise commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

SeaRise commented Aug 2, 2022

SeaRise commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

ti-chi-bot commented Aug 2, 2022

sre-bot commented Aug 2, 2022

Coverage for changed files

Coverage summary

Planner: planner without `DAGQueryBlock` #5381

Planner: planner without `DAGQueryBlock` #5381

SeaRise commented Jul 15, 2022 •

edited

Loading

ti-chi-bot commented Jul 15, 2022 •

edited

Loading