feat: pushdown get_format into TiFlash

[wirybeaver]

What problem does this PR solve?

Issue Number: close #5115
Related TiDB Issue: pingcap/tidb#36035

Problem Summary:
Implement GetFormat push down. Refer to the rule in the MySQL doc

Trigger manual integration test with the following command

➜  ~tiup playground --tiflash.binpath /Users/xuanyili/my-code/tiflash/build/dbms/src/Server/tiflash --db.binpath /Users/xuanyili/my-code/tidb/bin/tidb-server

The explain result shows that the expression get_format will be pushdown to TiFlash.

+---------------------------+----------+--------------+---------------+---------------------------------------------+
| id                        | estRows  | task         | access object | operator info                               |
+---------------------------+----------+--------------+---------------+---------------------------------------------+
| TableReader_9             | 10000.00 | root         |               | data:ExchangeSender_8                       |
| └─ExchangeSender_8        | 10000.00 | mpp[tiflash] |               | ExchangeType: PassThrough                   |
|   └─Projection_4          | 10000.00 | mpp[tiflash] |               | get_format(DATE, test.t.location)->Column#3 |
|     └─TableFullScan_7     | 10000.00 | mpp[tiflash] | table:t       | keep order:false, stats:pseudo              |
+---------------------------+----------+--------------+---------------+---------------------------------------------+

Unit Test

ninja gtests_dbms
./dbms/gtests_dbms --gtest_filter='GetFormatTest*'"

Integration Test

./run-test.sh fullstack-test/expr/get_format.test

What is changed and how it works?

• The first parameter is a MySQL reserved word without quote in the set of (DATETIME, DATE, TIME, TIMESTAMP). When the argument is push down to TiFlash, it should always be a ColumnConst. Thus, overwrite getArgumentsThatAreAlwaysConstant() by returning {0}.
• returns empty string in case of no rule matching according to the TiDB implementation.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Support get_format pushdown to TiFlash.

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• SeaRise
• zanmato1984

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[sre-bot]

All committers have signed the CLA.

[ti-chi-bot]

Welcome @wirybeaver!

It looks like this is your first PR to pingcap/tiflash 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/tiflash. 😃

[wirybeaver]

/run-integration-test

[SeaRise]

And add UT under dbms/src/Functions/tests

[SeaRise]

Add release note like 'Support to pushdown get_format to TiFlash'.
And add the pr link of tidb?

[wirybeaver]

And add UT under dbms/src/Functions/tests

Done

[wirybeaver]

Add release note like 'Support to pushdown get_format to TiFlash'. And add the pr link of tidb?

Done

[wirybeaver]

/run-all-tests

[zanmato1984]

git rebase OR git merge? Right now I use git commit --amend to keep my changes as a single commit and use git pull --rebase to fetch remote master change.

You are right. We don't have particular workflow for private branches. Squash merging into master is all.

[zanmato1984]

LGTM

[zanmato1984]

@wirybeaver Nice work and thanks for your contribution!

[SeaRise]

/merge

[ti-chi-bot]

@SeaRise: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[SeaRise]

/merge cancel

[sre-bot]

Coverage for changed files

Filename                                 Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp               347               155    55.33%          45                15    66.67%         638               294    53.92%         412               197    52.18%
Functions/FunctionsConversion.cpp             45                12    73.33%          10                 2    80.00%         171                22    87.13%          20                 7    65.00%
Functions/FunctionsConversion.h             1037               487    53.04%         153                75    50.98%        1776               879    50.51%         584               312    46.58%
Functions/tests/gtest_get_format.cpp         261                77    70.50%           2                 0   100.00%          74                 0   100.00%          78                41    47.44%
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                       1690               731    56.75%         210                92    56.19%        2659              1195    55.06%        1094               557    49.09%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18433      9624             47.79%    207477  96481        53.50%

full coverage report (for internal network access only)

[sre-bot]

Coverage for changed files

Filename                                 Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp               347               155    55.33%          45                15    66.67%         638               294    53.92%         412               197    52.18%
Functions/FunctionsConversion.cpp             45                12    73.33%          10                 2    80.00%         171                22    87.13%          20                 7    65.00%
Functions/FunctionsConversion.h             1037               487    53.04%         153                75    50.98%        1776               879    50.51%         584               312    46.58%
Functions/tests/gtest_get_format.cpp         261                77    70.50%           2                 0   100.00%          74                 0   100.00%          78                41    47.44%
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                       1690               731    56.75%         210                92    56.19%        2659              1195    55.06%        1094               557    49.09%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18433      9624             47.79%    207477  96502        53.49%

full coverage report (for internal network access only)

[wirybeaver]

@wirybeaver Nice work and thanks for your contribution!

Thanks for your guys comments and suggestions! Also a shout to @SeaRise

[wirybeaver]

/run-all-tests

[sre-bot]

Coverage for changed files

Filename                                 Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp               347               155    55.33%          45                15    66.67%         638               294    53.92%         412               197    52.18%
Functions/FunctionsConversion.cpp             45                12    73.33%          10                 2    80.00%         171                22    87.13%          20                 7    65.00%
Functions/FunctionsConversion.h             1037               487    53.04%         153                75    50.98%        1776               879    50.51%         584               312    46.58%
Functions/tests/gtest_get_format.cpp         261                77    70.50%           2                 0   100.00%          74                 0   100.00%          78                41    47.44%
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                       1690               731    56.75%         210                92    56.19%        2659              1195    55.06%        1094               557    49.09%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18449      9624             47.83%    207776  96544        53.53%

full coverage report (for internal network access only)

[wirybeaver]

@SeaRise Could you /merge this branch when you are available?

[Lloyd-Pottiger]

/merge

[ti-chi-bot]

@Lloyd-Pottiger: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 3f761ac

[ti-chi-bot]

@wirybeaver: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[sre-bot]

Coverage for changed files

Filename                                 Regions    Missed Regions     Cover   Functions  Missed Functions  Executed       Lines      Missed Lines     Cover    Branches   Missed Branches     Cover
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Flash/Coprocessor/DAGUtils.cpp               347               142    59.08%          45                11    75.56%         638               272    57.37%         412               192    53.40%
Functions/FunctionsConversion.cpp             45                12    73.33%          10                 2    80.00%         171                22    87.13%          20                 7    65.00%
Functions/FunctionsConversion.h             1037               483    53.42%         153                74    51.63%        1776               857    51.75%         584               310    46.92%
Functions/tests/gtest_get_format.cpp         261                77    70.50%           2                 0   100.00%          74                 0   100.00%          78                41    47.44%
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                       1690               714    57.75%         210                87    58.57%        2659              1151    56.71%        1094               550    49.73%

Coverage summary

Functions  MissedFunctions  Executed  Lines   MissedLines  Cover
18502      9567             48.29%    208642  96314        53.84%

full coverage report (for internal network access only)