Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tso: implement the global TSO generation algorithm #3033

Merged
merged 25 commits into from
Oct 14, 2020
Merged

tso: implement the global TSO generation algorithm #3033

merged 25 commits into from
Oct 14, 2020

Conversation

JmPotato
Copy link
Member

@JmPotato JmPotato commented Sep 27, 2020 •
edited
Loading

What problem does this PR solve?

Part of #2759. Resolve #2984. This PR includes two parts:

  • Implement the new global TSO generation algorithm, which includes about 200 lines of codes.
  • Refactor the TSO unit tests, which include about 300 lines of codes, but most of them are just structural adjustments.

The reason I put these two parts together is to have a CI test fully. If this brings you guys serious difficulty in reviewing, I will take these two parts separately though.

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Side effects

  • Possible performance regression
  • Increased code complexity

Release note

@JmPotato JmPotato added the component/tso Timestamp Oracle. label Sep 27, 2020
@JmPotato JmPotato self-assigned this Sep 27, 2020
disksing
disksing reviewed Sep 27, 2020
View reviewed changes
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
@JmPotato
Update the algorithm
d6dc7e2 
Signed-off-by: JmPotato <[email protected]>
@JmPotato
Copy link
Member Author

/run-all-tests

rleungx
rleungx reviewed Sep 28, 2020
View reviewed changes
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Show resolved Hide resolved
disksing
disksing reviewed Sep 28, 2020
View reviewed changes
server/grpc_service.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
@JmPotato
Copy link
Member Author

/run-all-tests

@JmPotato
Copy link
Member Author

/run-integration-common-test

@JmPotato JmPotato requested a review from rleungx October 9, 2020 03:05
@JmPotato
Resolve conflicts
b528c33 
Signed-off-by: JmPotato <[email protected]>
@JmPotato
Refine logs
000fe33 
Signed-off-by: JmPotato <[email protected]>
@JmPotato
Copy link
Member Author

/run-all-tests

disksing
disksing reviewed Oct 10, 2020
View reviewed changes
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/global_allocator.go Outdated Show resolved Hide resolved
server/tso/tso.go Show resolved Hide resolved
@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Oct 12, 2020
nolouch
nolouch reviewed Oct 13, 2020
View reviewed changes
Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reset LGTM.

server/tso/global_allocator.go
if currentGlobalTSO, err = gta.getCurrentTSO(); err != nil {
return pdpb.Timestamp{}, err
}
if tsoutil.CompareTimestamp(&currentGlobalTSO, maxTSO) < 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloud this update guarantee atomicity?

Copy link
Member Author

@JmPotato JmPotato Oct 13, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not atomic here. But even if the currentGlobalTSO changed during the comparison, the later SetTSO() will guarantee the atomicity and make sure the TSO won't fallback.

nolouch
nolouch reviewed Oct 14, 2020
View reviewed changes
Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot removed the status/LGT1 Indicates that a PR has LGTM 1. label Oct 14, 2020
@ti-srebot ti-srebot added the status/LGT2 Indicates that a PR has LGTM 2. label Oct 14, 2020
@nolouch
Copy link
Contributor

nolouch commented Oct 14, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Oct 14, 2020
@ti-srebot
Copy link
Contributor

/run-all-tests

@ti-srebot ti-srebot merged commit e0747f9 into tikv:master Oct 14, 2020
@JmPotato JmPotato deleted the global_local_tso_sync branch October 14, 2020 03:48
lhy1024 pushed a commit to lhy1024/pd that referenced this pull request Oct 15, 2020
@JmPotato @lhy1024
tso: implement the global TSO generation algorithm (tikv#3033)
32b9395 
Signed-off-by: JmPotato <[email protected]>
(cherry picked from commit e0747f9)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nolouch nolouch nolouch left review comments

@ti-srebot ti-srebot ti-srebot approved these changes

@disksing disksing disksing approved these changes

@Yisaer Yisaer Awaiting requested review from Yisaer

@rleungx rleungx Awaiting requested review from rleungx

Assignees

@JmPotato JmPotato

Labels
component/tso Timestamp Oracle. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement the global TSO generation algorithm
5 participants
@JmPotato @nolouch @ti-srebot @disksing @rleungx