util/ranger: fix wrong range calculation of prefix index when appending ranges to point ranges (#26066)

[ti-srebot]

cherry-pick #26066 to release-5.1
You can switch your code base to this Pull Request by using git-extras:

# In tidb repo:
git pr https://github.com/pingcap/tidb/pull/26262

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/tidb.git pr/26262:release-5.1-94ed411ab8bd

---

What problem does this PR solve?

Issue Number: close #26029

Problem Summary:
Query range of prefix index is wrong, which leads to wrong query result.

What is changed and how it works?

What's Changed & How it Works:
When building ranges for eq/in conditions, each range is a single point. If the index contains the prefix of some column, the ranges need be cut to the prefix on that column. After cutting on that column, it is possible to merge some overlapping or adjacent ranges so we call UnionRanges to do that. After calling UnionRanges, two adjacent single point ranges may be merged to one range. However, we still think the ranges consist of single points and use the ranges when calling appendRanges2PointRanges, which leads to wrong range calculation. For example, if the condition is col1 = "xxxxxx" and col2 in (2, 3) and col3 != 4 and the index is (col1(3), col2, col3), we build ["xxxxxx" 2, "xxxxxx" 2], ["xxxxxx" 3, "xxxxxx" 3] from eq/in conditions. After cutting on the first column, the ranges are ["xxx" 2, "xxx" 2], ["xxx" 3, "xxx" 3]. Then, after calling UnionRanges with rangeDetacher.mergeConsecutive = true, the ranges are ["xxx" 2, "xxx" 3]. Later we call appendRanges2PointRanges, the ranges are ["xxx" 2 -inf, "xxx" 3 4), ("xxx" 2 4, "xxx" 3 +inf], which are wrong.

In order to correct the wrong range calculation, we get pointRanges by calling rangeDetacher.buildCNFIndexRange with rangeDetacher.mergeConsecutive = false. In this way, two adjacent single point ranges would not be merged to one range in UnionRanges. So pointRanges consist of single points and we use pointRanges when calling appendRanges2PointRanges.

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:
• Need to cherry-pick to the release branch

Check List

Tests

• Unit test
• Integration test

Side effects

• Performance regression
  • Consumes more CPU
  • Consumes more MEM

Release note

• fix a bug on the query range of prefix index

[ti-srebot]

/run-all-tests

[ti-srebot]

@xuyifangreeneyes you're already a collaborator in bot's repo.

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• qw4990
• winoros

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[guo-shaoge]

/run-common-test tidb-test=pr/1244
/run-integration-common-test tidb-test=pr/1244

[zhouqiang-cl]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: e6edeba

[ti-chi-bot]

@ti-srebot: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.