*: support dynamic tso service #8517

rleungx · 2024-08-12T08:09:05Z

What problem does this PR solve?

Issue Number: ref #8477

What is changed and how does it work?

Check List

Tests

Unit test

Release note

None.

JmPotato · 2024-08-15T05:32:38Z

client/client.go

@@ -660,7 +660,7 @@ func (c *client) Close() {
 	}
 }

-func (c *client) setServiceMode(newMode pdpb.ServiceMode) {
+func (c *client) setServiceMode(newMode pdpb.ServiceMode, skipSameMode bool) {


Prefer using a more straightforward word.

Suggested change

func (c *client) setServiceMode(newMode pdpb.ServiceMode, skipSameMode bool) {

func (c *client) setServiceMode(newMode pdpb.ServiceMode, force bool) {

It's not the same as force.

why not skipSameMode always?

JmPotato · 2024-08-15T05:34:50Z

client/errs/errs.go

+	}
+	errMsg := err.Error()
+	return strings.Contains(errMsg, "not found tso address") ||
+		strings.Contains(errMsg, "maximum number of retries exceeded")


This error would also occur when the leadership cannot be elected. In which case will this be a misjudgment?

We don't check this error on client side, do you know about the reason?

JmPotato · 2024-08-15T05:37:00Z

tests/integrations/client/client_test.go

@@ -406,6 +406,8 @@ func TestTSOFollowerProxyWithTSOService(t *testing.T) {
 	backendEndpoints := pdLeaderServer.GetAddr()
 	tsoCluster, err := tests.NewTestTSOCluster(ctx, 2, backendEndpoints)
 	re.NoError(err)
+	// let service discovery know the TSO service
+	time.Sleep(500 * time.Millisecond)


Can it be replaced with an Eventually?

codecov · 2024-08-21T06:44:34Z

Codecov Report

Attention: Patch coverage is 67.07317% with 54 lines in your changes missing coverage. Please review.

Project coverage is 77.48%. Comparing base (5ba4db7) to head (63c78f8).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8517      +/-   ##
==========================================
- Coverage   77.53%   77.48%   -0.05%     
==========================================
  Files         474      474              
  Lines       62355    62469     +114     
==========================================
+ Hits        48345    48404      +59     
- Misses      10437    10485      +48     
- Partials     3573     3580       +7

Flag	Coverage Δ
unittests	`77.48% <67.07%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

okJiang · 2024-08-29T06:40:09Z

client/errs/errs.go

@@ -38,6 +38,16 @@ func IsLeaderChange(err error) bool {
 		strings.Contains(errMsg, NotPrimaryErr)
 }

+// IsServiceModeChange will determine whether there is a service mode change.


Suggested change

// IsServiceModeChange will determine whether there is a service mode change.

// IsServiceModeChange determines whether there is a service mode change.

okJiang · 2024-08-29T06:46:26Z

server/forward.go

-		if err != nil {
-			if needRetry := handleStreamError(err); needRetry {
-				continue
+	if s.forwardToTSOService() {


We can reduce some of the indentation

Suggested change

if s.forwardToTSOService() {

if !s.forwardToTSOService() {

return s.tsoAllocatorManager.HandleRequest(ctx, tso.GlobalDCLocation, 1)

}

request := xxxx

.....

okJiang · 2024-08-29T07:51:22Z

server/grpc_service.go

@@ -569,6 +590,72 @@ func (s *GrpcServer) Tso(stream pdpb.PD_TsoServer) error {
 			continue
 		}

+		if s.forwardToTSOService() {


maybe wrap three situation(local tso/ms tso/normal) to three functions. Just a suggestion.

okJiang · 2024-08-29T08:03:10Z

server/cluster/cluster.go

+			if !c.IsServiceIndependent(constant.TSOServiceName) {
+				// leader tso service exit, tso independent service provide tso
+				c.tsoAllocator.ResetAllocatorGroup(tso.GlobalDCLocation, true)
+			}
+			if !c.IsServiceIndependent(constant.TSOServiceName) {
+				log.Info("TSO server starts to provide timestamp")
+			}
+			c.SetServiceIndependent(constant.TSOServiceName)


Suggested change

if !c.IsServiceIndependent(constant.TSOServiceName) {

// leader tso service exit, tso independent service provide tso

c.tsoAllocator.ResetAllocatorGroup(tso.GlobalDCLocation, true)

}

if !c.IsServiceIndependent(constant.TSOServiceName) {

log.Info("TSO server starts to provide timestamp")

}

c.SetServiceIndependent(constant.TSOServiceName)

if !c.IsServiceIndependent(constant.TSOServiceName) {

// leader tso service exit, tso independent service provide tso

c.tsoAllocator.ResetAllocatorGroup(tso.GlobalDCLocation, true)

log.Info("TSO server starts to provide timestamp")

}

c.SetServiceIndependent(constant.TSOServiceName)

okJiang · 2024-08-29T08:15:15Z

server/cluster/cluster.go

@@ -390,24 +397,84 @@ func (c *RaftCluster) checkServices() {
 	}
 }

+// checkTSOService checks the TSO service.
+func (c *RaftCluster) checkTSOService() {


can you add more comments for this function and inside this function. Seems there are too many situations in this function

okJiang · 2024-08-29T08:17:58Z

client/pd_service_discovery.go


 	ctx, cancel := context.WithCancel(c.ctx)
 	defer cancel()
 	ticker := time.NewTicker(serviceModeUpdateInterval)
+	failpoint.Inject("fastUpdateServiceMode", func() {
+		ticker.Stop()
+		ticker = time.NewTicker(10 * time.Millisecond)


ticker.Reset()?

okJiang · 2024-08-29T08:27:57Z

client/client.go

@@ -713,6 +712,7 @@ func (c *client) resetTSOClientLocked(mode pdpb.ServiceMode) {
 		log.Warn("[pd] intend to switch to unknown service mode, just return")
 		return
 	}
+	// Replace the old TSO client.


okJiang · 2024-08-29T08:30:09Z

client/client.go

@@ -660,7 +660,7 @@ func (c *client) Close() {
 	}
 }

-func (c *client) setServiceMode(newMode pdpb.ServiceMode) {
+func (c *client) setServiceMode(newMode pdpb.ServiceMode, skipSameMode bool) {


why not skipSameMode always?

ti-chi-bot · 2024-09-06T08:36:06Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Signed-off-by: Ryan Leung <[email protected]>

ti-chi-bot · 2024-09-29T10:02:23Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rleungx and additionally assign yudongusa for approval(Please ensuring that each of them provides their approval before proceeding). For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS
server/config/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Signed-off-by: Ryan Leung <[email protected]>

ti-chi-bot bot added do-not-merge/needs-linked-issue release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates the PR's author has signed the dco. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 12, 2024

rleungx requested review from JmPotato, nolouch and HuSharp August 14, 2024 08:19

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 15, 2024

JmPotato reviewed Aug 15, 2024

View reviewed changes

rleungx force-pushed the dynamic-switch branch from 6c98636 to dc6b4a7 Compare August 21, 2024 06:36

rleungx force-pushed the dynamic-switch branch 2 times, most recently from a36ad5d to f2c0c14 Compare August 23, 2024 07:38

ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 26, 2024

rleungx force-pushed the dynamic-switch branch from 5d11416 to 12c7821 Compare August 27, 2024 06:30

ti-chi-bot bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 27, 2024

okJiang reviewed Aug 29, 2024

View reviewed changes

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 2, 2024

rleungx force-pushed the dynamic-switch branch from 96c89a1 to 0a659c8 Compare September 9, 2024 08:05

rleungx requested review from JmPotato and okJiang September 9, 2024 08:05

rleungx force-pushed the dynamic-switch branch from 0a659c8 to 80133c9 Compare September 10, 2024 07:17

rleungx added 2 commits September 29, 2024 17:15

support dynamic tso service

7a96b22

Signed-off-by: Ryan Leung <[email protected]>

add a test

8d59ada

Signed-off-by: Ryan Leung <[email protected]>

rleungx added 18 commits September 29, 2024 17:17

fix

f15f214

Signed-off-by: Ryan Leung <[email protected]>

fix

f2d7b4a

Signed-off-by: Ryan Leung <[email protected]>

fix

8d1cfa2

Signed-off-by: Ryan Leung <[email protected]>

fix test

693cbac

Signed-off-by: Ryan Leung <[email protected]>

fix test

a7bfd4e

Signed-off-by: Ryan Leung <[email protected]>

fix test

2e6574d

Signed-off-by: Ryan Leung <[email protected]>

tiny change

d7661e2

Signed-off-by: Ryan Leung <[email protected]>

tiny change

60eb4f4

Signed-off-by: Ryan Leung <[email protected]>

discover faster

40639be

Signed-off-by: Ryan Leung <[email protected]>

forward request without recreating stream

419dbc4

Signed-off-by: Ryan Leung <[email protected]>

fix

1b71207

Signed-off-by: Ryan Leung <[email protected]>

fix

362a68a

Signed-off-by: Ryan Leung <[email protected]>

fix

dcc233a

Signed-off-by: Ryan Leung <[email protected]>

fix failpoint

ce05cda

Signed-off-by: Ryan Leung <[email protected]>

fix

e24f1ad

Signed-off-by: Ryan Leung <[email protected]>

address comments

a73ebe8

Signed-off-by: Ryan Leung <[email protected]>

address the comment

0d4b723

Signed-off-by: Ryan Leung <[email protected]>

resolve conflicts

e3f29e6

Signed-off-by: Ryan Leung <[email protected]>

rleungx force-pushed the dynamic-switch branch from 80133c9 to e3f29e6 Compare September 29, 2024 09:43

ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. do-not-merge/needs-linked-issue labels Sep 29, 2024

rleungx removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 29, 2024

remove unstable test

63c78f8

Signed-off-by: Ryan Leung <[email protected]>

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 29, 2024

add config test

3d9861a

Signed-off-by: Ryan Leung <[email protected]>

ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: support dynamic tso service #8517

*: support dynamic tso service #8517

rleungx commented Aug 12, 2024 •

edited

Loading

JmPotato Aug 15, 2024

rleungx Aug 23, 2024

okJiang Aug 29, 2024

JmPotato Aug 15, 2024

rleungx Aug 23, 2024

JmPotato Aug 15, 2024

codecov bot commented Aug 21, 2024 •

edited

Loading

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

okJiang Aug 29, 2024

ti-chi-bot bot commented Sep 6, 2024

ti-chi-bot bot commented Sep 29, 2024

	func (c *client) setServiceMode(newMode pdpb.ServiceMode, skipSameMode bool) {
	func (c *client) setServiceMode(newMode pdpb.ServiceMode, force bool) {

	// IsServiceModeChange will determine whether there is a service mode change.
	// IsServiceModeChange determines whether there is a service mode change.

-	if s.forwardToTSOService() {
+	if !s.forwardToTSOService() {
+        return s.tsoAllocatorManager.HandleRequest(ctx, tso.GlobalDCLocation, 1)
+    }
+    request := xxxx
+    .....

*: support dynamic tso service #8517

Are you sure you want to change the base?

*: support dynamic tso service #8517

Conversation

rleungx commented Aug 12, 2024 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 21, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ti-chi-bot bot commented Sep 6, 2024

ti-chi-bot bot commented Sep 29, 2024

rleungx commented Aug 12, 2024 •

edited

Loading

codecov bot commented Aug 21, 2024 •

edited

Loading