-
Notifications
You must be signed in to change notification settings - Fork 222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DNM] support GetRegion via CSE sync_region API
#745
Open
iosmanthus
wants to merge
59
commits into
master
Choose a base branch
from
cse-region-client
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 21 commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
97cea40
refactor fanout logic of cse region client
iosmanthus 4da4647
add todos about region info resp from cse
iosmanthus 7ab0132
support get store for CSEClient
iosmanthus de2f0f4
update comment for cse test
iosmanthus 5bd7f71
remove logs
iosmanthus 4054b80
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus 05b48d2
add store prober
iosmanthus e05be61
fix compile
iosmanthus a9ad5f5
remove logs
iosmanthus e3a2ba2
add cb for Fallback and CSEClient
iosmanthus 70cc79e
goimports -w ./
iosmanthus f8201f7
introduce asyncBreaker for Fallback and CSEClient
iosmanthus e2fdd40
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus ccbfa16
bump kvproto
iosmanthus bca479e
goimports -w ./
iosmanthus f6b9f97
fix lint
iosmanthus e8cf959
go mod tidy for integration tests
iosmanthus 8b68569
introduce EnableCSERegionClient for TiKVClient
iosmanthus 770e723
refine interface comments and options
iosmanthus 01caebd
bump kvproto to master
iosmanthus 055f177
Merge branch 'master' into cse-region-client
iosmanthus e1b0313
execdetails: add RRU/WRU interface of RURuntimeStats (#768)
nolouch 6de096c
fix encoding of mvcc get by key (#775)
iosmanthus 94baaf7
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus 4011b67
fix panic while collects buckets (#774)
iosmanthus f53c17d
add const label for metrics (#781)
zeminzhou fcf9595
force to sync the current region while GetPrevRegion (#783)
iosmanthus 716c8a5
encode store batch tasks (#785)
iosmanthus 4d2f943
cse: decode batch cop task (#798)
iosmanthus 31152f5
*: Add GetMinTS (#801)
rleungx d5baf4c
Use a new way to create client for v2 (#806)
rleungx a2f93a3
Count only one tiflash replica in disaggregated mode. (#805)
JinheLin 5f10ed1
cse: catch up master (#810)
iosmanthus 35937f2
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus d8ce633
Merge branch 'cse-region-client' of github.com:tikv/client-go into cs…
iosmanthus a6a7be2
Add get min ts implementation (#809)
rleungx 1d62208
*: fix the lint and add CI (#813)
rleungx 388baf3
keyspace gc cse client (#803)
ystaticy 4499cc5
support remote coprocessor (#808)
coocood a07600f
cse: add replica number (#824)
nolouch 390b11b
add logs for cse region client (#838)
iosmanthus d2d449f
filter out removed node state (#849)
iosmanthus a246fae
polish comments for cse region client (#850)
iosmanthus 136f833
*: update pd client version (#840)
lhy1024 d5a7044
refactor cse region client as plugin (#853)
iosmanthus 43fcadf
rebase master for cse-region-client (#854)
iosmanthus 1d58034
fix conflicts with master for cse-region-client (#856)
iosmanthus f04046a
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus 19ef437
resource_control: bypass some internal urgent request (#884)
nolouch c22bb29
pdclient: get all keyspace (#892)
ystaticy b27cb07
resourcecontrol: fix nil pointer (#900)
nolouch 2ad441f
prefix safepoint kv with keyspace name (#928)
AmoebaProtozoa b67add4
merge master into cse region client (#939)
iosmanthus e93b07c
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus a29e95d
collecting the RU information by pasing point through context.Value (…
zeminzhou e1d4f84
increase large transaction preSplitSizeThreashold (#1059)
coocood 0041484
merge master for cse region client (#1083)
iosmanthus 4ffb53a
Merge branch 'master' of github.com:tikv/client-go into cse-region-cl…
iosmanthus aaa66ef
Merge branch 'master' into cse-region-client
iosmanthus File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| package tikv_test | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/hex" | ||
| "log" | ||
| "strings" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/suite" | ||
| "github.com/tikv/client-go/v2/tikv" | ||
| pd "github.com/tikv/pd/client" | ||
| ) | ||
|
|
||
| // TODO(iosmanthus): refactor this suite as a unit test. | ||
| func TestCSE(t *testing.T) { | ||
| if !*withCSE { | ||
| t.Skip("skipping test; use -with-cse flag to run this test") | ||
| } | ||
| suite.Run(t, new(cseSuite)) | ||
| } | ||
|
|
||
| type cseSuite struct { | ||
| suite.Suite | ||
| pdCli pd.Client | ||
| } | ||
|
|
||
| func (s *cseSuite) SetupTest() { | ||
| pdCli, err := pd.NewClient(strings.Split(*pdAddrs, ","), pd.SecurityOption{}) | ||
| s.Nil(err) | ||
| pdCli, err = tikv.NewCSEClient(pdCli, nil) | ||
| s.Nil(err) | ||
| s.pdCli = tikv.NewCodecPDClient(tikv.ModeTxn, pdCli) | ||
| } | ||
|
|
||
| func (s *cseSuite) TearDownTest() { | ||
| s.pdCli.Close() | ||
| } | ||
|
|
||
| func (s *cseSuite) TestGetRegion() { | ||
| key, err := hex.DecodeString("780000016d44444c5461626c65ff56657273696f6e00fe0000000000000073") | ||
| s.Nil(err) | ||
| currentRegion, err := s.pdCli.GetRegion(context.Background(), key) | ||
| s.Nil(err) | ||
| s.NotNil(currentRegion) | ||
| s.LessOrEqual(currentRegion.Meta.StartKey, key) | ||
| s.Less(key, currentRegion.Meta.EndKey) | ||
| prevRegion, err := s.pdCli.GetPrevRegion(context.Background(), key) | ||
| s.Nil(err) | ||
| s.Equal(prevRegion.Meta.EndKey, currentRegion.Meta.StartKey) | ||
| } | ||
|
|
||
| func BenchmarkGetRegionByPD(b *testing.B) { | ||
| pdCli, err := pd.NewClient(strings.Split(*pdAddrs, ","), pd.SecurityOption{}) | ||
| if err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| pdCli = tikv.NewCodecPDClient(tikv.ModeTxn, pdCli) | ||
| b.ResetTimer() | ||
| for i := 0; i < b.N; i++ { | ||
| region, err := pdCli.GetRegion(context.Background(), []byte("780000016d44444c5461626c65ff56657273696f6e00fe0000000000000073")) | ||
| if err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| if region == nil { | ||
| log.Fatalln("region is nil") | ||
| } | ||
| } | ||
| b.StopTimer() | ||
| pdCli.Close() | ||
| } | ||
|
|
||
| func BenchmarkGetRegionByCSE(b *testing.B) { | ||
| pdCli, err := pd.NewClient(strings.Split(*pdAddrs, ","), pd.SecurityOption{}) | ||
| if err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| pdCli, err = tikv.NewCSEClient(pdCli, nil) | ||
| if err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| pdCli = tikv.NewCodecPDClient(tikv.ModeTxn, pdCli) | ||
| b.ResetTimer() | ||
| for i := 0; i < b.N; i++ { | ||
| region, err := pdCli.GetRegion(context.Background(), []byte("780000016d44444c5461626c65ff56657273696f6e00fe0000000000000073")) | ||
| if err != nil { | ||
| b.Fatal(err) | ||
| } | ||
| if region == nil { | ||
| log.Fatalln("region is nil") | ||
| } | ||
| } | ||
| b.StopTimer() | ||
| pdCli.Close() | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we also test
GetRegionByID,ScanRegionsandGetPrevRegion?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test could not run via CI currently, we might complete the integration tests/unit tests later.