Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

support last_day and dayofmonth pushdown to tiflash #4183

Merged
merged 20 commits into from
Mar 14, 2022
Merged

support last_day and dayofmonth pushdown to tiflash #4183

merged 20 commits into from
Mar 14, 2022

Conversation

guo-shaoge
Copy link
Contributor

@guo-shaoge guo-shaoge commented Mar 6, 2022 •
edited
Loading

Signed-off-by: guo-shaoge [email protected]

What problem does this PR solve?

Issue Number: close #4149

Problem Summary:

According MySQL doc: DAY() is a synonym for DAYOFMONTH().

So DayOfMonth will just use Day implementation.

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

support last_day and dayofmonth pushdown to tiflash

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 6, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • XuHuaiyu
  • yibin87

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed do-not-merge/needs-linked-issue labels Mar 6, 2022
@guo-shaoge
fix fmt
5b1ba88 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@ti-chi-bot ti-chi-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Mar 6, 2022
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 6, 2022

Coverage for changed files

Filename                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGContext.h                       33                10    69.70%          26                 9    65.38%          74                17    77.03%           8                 3    62.50%
Flash/Coprocessor/DAGUtils.cpp                      285               228    20.00%          35                23    34.29%         491               390    20.57%         320               226    29.38%
Functions/FunctionsDateTime.cpp                      42                20    52.38%           5                 2    60.00%         122                31    74.59%          26                14    46.15%
Functions/FunctionsDateTime.h                      1086               638    41.25%         322               208    35.40%        2024              1227    39.38%         548               343    37.41%
Functions/tests/gtest_datetime_last_day.cpp         114                34    70.18%           1                 0   100.00%          37                 0   100.00%          34                18    47.06%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                              1560               930    40.38%         389               242    37.79%        2748              1665    39.41%         936               604    35.47%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16722      9456             43.45%    187327  95581        48.98%

full coverage report (for internal network access only)

@guo-shaoge
comment some test
e8bf560 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
Copy link
Contributor Author

/cc @windtalker

@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 7, 2022

Coverage for changed files

Filename                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGContext.h                       33                10    69.70%          26                 9    65.38%          74                17    77.03%           8                 3    62.50%
Flash/Coprocessor/DAGUtils.cpp                      285               228    20.00%          35                23    34.29%         491               390    20.57%         320               226    29.38%
Functions/FunctionsDateTime.cpp                      42                20    52.38%           5                 2    60.00%         122                31    74.59%          26                14    46.15%
Functions/FunctionsDateTime.h                      1086               638    41.25%         322               208    35.40%        2024              1227    39.38%         548               343    37.41%
Functions/tests/gtest_datetime_last_day.cpp         114                34    70.18%           1                 0   100.00%          37                 0   100.00%          34                18    47.06%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                              1560               930    40.38%         389               242    37.79%        2748              1665    39.41%         936               604    35.47%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16722      9456             43.45%    187327  95576        48.98%

full coverage report (for internal network access only)

@guo-shaoge
cleaner code and more case
71ff88e 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
fix
8d929b8 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
fix comment
0b8f880 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
Merge branch 'last_day' of github.com:guo-shaoge/tics into last_day
fb02a3e 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
fix fmt
ad2a4cc 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 14, 2022

Coverage for changed files

Filename                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/MyTime.cpp                                   985               194    80.30%         107                25    76.64%        2036               481    76.38%         872               238    72.71%
Common/MyTime.h                                      22                 0   100.00%           9                 0   100.00%          30                 0   100.00%          16                 3    81.25%
Flash/Coprocessor/DAGContext.h                       33                15    54.55%          26                14    46.15%          74                32    56.76%           8                 3    62.50%
Flash/Coprocessor/DAGUtils.cpp                      290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Functions/FunctionsDateTime.cpp                      27                19    29.63%           4                 2    50.00%          98                25    74.49%          16                13    18.75%
Functions/FunctionsDateTime.h                      1070               637    40.47%         319               208    34.80%        2007              1223    39.06%         534               340    36.33%
Functions/tests/gtest_datetime_last_day.cpp         137                37    72.99%           1                 0   100.00%          65                 0   100.00%          42                21    50.00%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                              2564              1135    55.73%         502               273    45.62%        4815              2165    55.04%        1812               848    53.20%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16870      9462             43.91%    189749  95874        49.47%

full coverage report (for internal network access only)

@guo-shaoge
fix fmt
fc15e4d 
Signed-off-by: guo-shaoge <[email protected]>
@guo-shaoge
Copy link
Contributor Author

/run-all-tests

@sre-bot
Copy link
Collaborator

sre-bot commented Mar 14, 2022

Coverage for changed files

Filename                                        Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Common/MyTime.cpp                                   985               194    80.30%         107                25    76.64%        2036               481    76.38%         872               238    72.71%
Common/MyTime.h                                      22                 0   100.00%           9                 0   100.00%          30                 0   100.00%          16                 3    81.25%
Flash/Coprocessor/DAGContext.h                       33                15    54.55%          26                14    46.15%          74                32    56.76%           8                 3    62.50%
Flash/Coprocessor/DAGUtils.cpp                      290               233    19.66%          36                24    33.33%         505               404    20.00%         324               230    29.01%
Functions/FunctionsDateTime.cpp                      27                19    29.63%           4                 2    50.00%          98                25    74.49%          16                13    18.75%
Functions/FunctionsDateTime.h                      1070               637    40.47%         319               208    34.80%        2007              1223    39.06%         534               340    36.33%
Functions/tests/gtest_datetime_last_day.cpp         137                37    72.99%           1                 0   100.00%          65                 0   100.00%          42                21    50.00%
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                              2564              1135    55.73%         502               273    45.62%        4815              2165    55.04%        1812               848    53.20%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16870      9462             43.91%    189749  95886        49.47%

full coverage report (for internal network access only)

yibin87
yibin87 approved these changes Mar 14, 2022
View reviewed changes
Copy link
Contributor

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Mar 14, 2022
XuHuaiyu
XuHuaiyu approved these changes Mar 14, 2022
View reviewed changes
Copy link
Contributor

@XuHuaiyu XuHuaiyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Mar 14, 2022
@guo-shaoge
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

@guo-shaoge: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: fc15e4d

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Mar 14, 2022
@sre-bot
Copy link
Collaborator

sre-bot commented Mar 14, 2022

Coverage for changed files

too many lines from llvm-cov, please refer to full report instead

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
16907      9490             43.87%    190388  96252        49.44%

full coverage report (for internal network access only)

@ti-chi-bot ti-chi-bot merged commit c7d69d4 into pingcap:master Mar 14, 2022
ywqzzy pushed a commit to ywqzzy/tiflash_1 that referenced this pull request Mar 14, 2022
@guo-shaoge @ywqzzy
support last_day and dayofmonth pushdown to tiflash (pingcap#4183)
cc2b968 
close pingcap#4149

Signed-off-by: ywqzzy <[email protected]>
ywqzzy added a commit to ywqzzy/tiflash_1 that referenced this pull request Mar 14, 2022
@ywqzzy
update.
b54dece 
Signed-off-by: ywqzzy <[email protected]>

update.

Signed-off-by: ywqzzy <[email protected]>

update.

Signed-off-by: ywqzzy <[email protected]>

Split write and ingest throughput (pingcap#4245)

close pingcap#4243

Signed-off-by: ywqzzy <[email protected]>

fix the problem that expired data was not recycled timely due to slow gc speed (pingcap#4224)

close pingcap#4146

Signed-off-by: ywqzzy <[email protected]>

PageStorage: Fix some bugs (pingcap#4212)

ref pingcap#3594

Signed-off-by: ywqzzy <[email protected]>

Make test of CreateTables using affected opts (pingcap#4239)

close pingcap#4235

Signed-off-by: ywqzzy <[email protected]>

BlobStore: remove the old_ids, not need recycle the blob id. (pingcap#4247)

ref pingcap#3594

Signed-off-by: ywqzzy <[email protected]>

support `PartitionTableScan` in TiFlash (pingcap#3876)

close pingcap#3873

Signed-off-by: ywqzzy <[email protected]>

update.

Signed-off-by: ywqzzy <[email protected]>

update.

Signed-off-by: ywqzzy <[email protected]>

support last_day and dayofmonth pushdown to tiflash (pingcap#4183)

close pingcap#4149

Signed-off-by: ywqzzy <[email protected]>
@guo-shaoge guo-shaoge mentioned this pull request Mar 17, 2022
Closed
@JaySon-Huang JaySon-Huang deleted the last_day branch March 17, 2022 02:30
JaySon-Huang pushed a commit to JaySon-Huang/tiflash that referenced this pull request Mar 17, 2022
@guo-shaoge @JaySon-Huang
support last_day and dayofmonth pushdown to tiflash (pingcap#4183)
1e07fab 
close pingcap#4149
@guo-shaoge guo-shaoge mentioned this pull request Mar 17, 2022
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@XuHuaiyu XuHuaiyu XuHuaiyu approved these changes

@yibin87 yibin87 yibin87 approved these changes

@windtalker windtalker Awaiting requested review from windtalker

Assignees
No one assigned
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support dayofmonth and last_day in TiFlash
5 participants
@guo-shaoge @ti-chi-bot @sre-bot @XuHuaiyu @yibin87