executor: fix load data losing connection when batch_dml_size is set (#22724)

[ti-srebot]

cherry-pick #22724 to release-4.0
You can switch your code base to this Pull Request by using git-extras:

# In tidb repo:
git pr https://github.com/pingcap/tidb/pull/22736

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/tidb.git pr/22736:release-4.0-8ddaf03db14a

---

What problem does this PR solve?

Issue Number: Fix #22540

Problem Summary:
When tidb_dml_batch_size is a relatively small(eg: 128/64), load data into a table which has a auto_random column may got lost connection error.

What is changed and how it works?

What's Changed:

1. add InsertValues::isLoadData member
2. add InsertValues::txnInUse (a mutex) member
3. lock txnInUse when commit routine needs to refresh txn ctx
4. lock txnInUse when process stream routine needs to generate auto random value

How it Works:

load data use two routine:

1. process stream routine: parse file and generate batch insert task
2. commit routine: read task from channel and do commit

Commit routine will invalid txn after task is committed. And before generate a new txn, process stream routine may use that invalid txn to generate audo random value. So we add a lock to protect txn, make sure process stream routine use a valid txn.

Related changes

• Need to cherry-pick to the release branch

Check List

Tests

• Unit test
• Integration test
  • TestLoadDataAutoRandom
• Manual test (add detailed scripts or steps below)
  1. generate csv files(5W rows)
  2. set @@session.tidb_dml_batch_size = 128;
  3. drop table if exists t;
  4. create table t(c1 bigint auto_random primary key, c2 bigint, c3 bigint);
  5. load data local infile %q into table t (c2, c3);
• No code

Side effects

• Performance regression
  • add a lock when commit task and generate auto random, may got lock conflict and slow down load data performance.

Release note

• fix load data lost connection error on tables with auto_random column

[ti-srebot]

/run-all-tests

[ti-srebot]

@guo-shaoge please accept the invitation then you can push to the cherry-pick pull requests.
https://github.com/ti-srebot/tidb/invitations

[guo-shaoge]

@AilinKid @XuHuaiyu

[AilinKid]

LGTM

[XuHuaiyu]

lgtm

[XuHuaiyu]

/merge

[ti-srebot]

Sorry @XuHuaiyu, this branch's release version is in progress, please contact zhouqiang-cl,shuke987,jebter,you06 for more details.

[guo-shaoge]

@XuHuaiyu

[XuHuaiyu]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 88b8ae1

[ti-chi-bot]

@ti-srebot: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.