Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

pkg/lightning: replace local generated ts with tso fetch from pd (#850) #860

Merged
merged 2 commits into from
Mar 12, 2021

Conversation

ti-srebot
Copy link
Contributor

@ti-srebot ti-srebot commented Mar 12, 2021

cherry-pick #850 to release-4.0
You can switch your code base to this Pull Request by using git-extras:

# In br repo:
git pr https://github.com/pingcap/br/pull/860

After apply modifications, you can push your change to this PR via:

git push [email protected]:ti-srebot/br.git pr/860:release-4.0-8a3f5e230714

What problem does this PR solve?

Replace commit ts generated with local time by tso fetch from pd. This can address the issue that if local time is mush faster than tidb cluster time, after import data with local/importer backend, query may returns empty result.

What is changed and how it works?

  • Fetch a ts from pd and store it in RestoreController and use this ts for all WriteRows
  • Always fetch the newest ts from pd when do table checksum. This is useful when we support concurrent import with many lightnings and only do checksum after the last lightning finished.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release Note

  • Fix the bug that lightning generated ts may be to large or small that query may return incorrect result.

@overvenus
Copy link
Member

LGTM

@ti-srebot ti-srebot added the status/LGT1 LGTM1 label Mar 12, 2021
@overvenus overvenus added this to the v4.0.12 milestone Mar 12, 2021
@kennytm
Copy link
Collaborator

kennytm commented Mar 12, 2021

/lgtm

@ti-srebot ti-srebot added status/LGT2 LGTM2 and removed status/LGT1 LGTM1 labels Mar 12, 2021
@kennytm
Copy link
Collaborator

kennytm commented Mar 12, 2021

/merge

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot ti-srebot merged commit d43d487 into pingcap:release-4.0 Mar 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants