feat: Broadcast min_commit_ts for pipelined transactions

[ekexium]

Ref: tikv/tikv#17459
Prerequires pingcap/kvproto#1265

Changes

1. Periodic updates of min_commit_ts
2. Broadcast min_commit_ts
3. Periodic updates of full store list

[you06]

BTW, this is not related to this PR, can you change the RLock/RUnlock to Lock/Unlock here?

I have a question about the future plan, have you considered apply this broadcast also for slow or large 2pc transactions?

[ekexium]

BTW, this is not related to this PR, can you change the RLock/RUnlock to Lock/Unlock here?

Done.

I have a question about the future plan, have you considered apply this broadcast also for slow or large 2pc transactions?

It's possible. But I think we need more empirical data before implementing that. It's unclear now how this optimization would benefit normal large transactions.

[cfzjywxk]

Better to explain in the comments when the pdClient.GetAllStores(ctx) may return a store list containg nil store and emptry address and how should they be treated. For example, dose it mean the returned store list is invalid and pdClient.GetAllStores(ctx) should be retried when there invalid element in the list?

[ekexium]

I believe that GetAllStores is supposed to return only stores in Up and Offline states. This additional check serves as a defensive measure and ensures consistency with the store resolve procedure

[zyguan]

Since we've already used a goroutine pool, how about spawning broadcast tasks like the following:

	stores := store.GetRegionCache().GetStoresByType(tikvrpc.TiKV)
	rateLimit := make(chan struct{}, min(broadcastMaxConcurrency, len(stores)))
	for _, s := range stores {
		rateLimit <- struct{}{}
		target := s
		err := txn.spawnWithStorePool(func() {
			defer func() {
				<-rateLimit
			}()
			// send request to target
		})
		if err != nil {
			<-rateLimit
		}
	}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, zyguan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cfzjywxk,zyguan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-09-24 12:53:30.169772239 +0000 UTC m=+1570479.910196175: ☑️ agreed by zyguan.
• 2024-09-25 01:31:10.938538581 +0000 UTC m=+1615940.678962520: ☑️ agreed by cfzjywxk.