Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

restore: Adapt the protocol of caching storage #39541

Merged
merged 11 commits into from
Dec 1, 2022
Merged

restore: Adapt the protocol of caching storage #39541

Changes from 9 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
3f292bd
apply cache
YuJuncen Oct 28, 2022
a6cb21e
patch kv proto
YuJuncen Nov 4, 2022
c0529a8
Merge branch 'master' of https://github.com/pingcap/tidb into cache-s…
YuJuncen Nov 4, 2022
faf4167
Merge branch 'master' of https://github.com/pingcap/tidb into cache-s…
YuJuncen Nov 10, 2022
29cd74c
use default kvproto
YuJuncen Dec 1, 2022
3559cf2
Merge branch 'master' of https://github.com/pingcap/tidb into cache-s…
YuJuncen Dec 1, 2022
43ae7c8
remove debug
YuJuncen Dec 1, 2022
0d02b32
move cache key into FileImporter
YuJuncen Dec 1, 2022
b8c52a5
restore go.mod
YuJuncen Dec 1, 2022
498cd63
Merge branch 'master' into cache-storage
ti-chi-bot Dec 1, 2022
73d8510
Merge branch 'master' into cache-storage
YuJuncen Dec 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions br/pkg/restore/import.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ import (
"bytes"
"context"
"crypto/tls"
"fmt"
"math/rand"
"strings"
"sync"
"sync/atomic"
Expand Down Expand Up @@ -247,6 +249,8 @@ type FileImporter struct {
rawStartKey []byte
rawEndKey []byte
supportMultiIngest bool

cacheKey string
}

// NewFileImporter returns a new file importClient.
Expand All @@ -261,6 +265,7 @@ func NewFileImporter(
backend: backend,
importClient: importClient,
isRawKvMode: isRawKvMode,
cacheKey: fmt.Sprintf("BR-%s-%d", time.Now().Format("20060102150405"), rand.Int63()),
}
}

Expand Down Expand Up @@ -636,6 +641,7 @@ func (importer *FileImporter) downloadSST(
Name: file.GetName(),
RewriteRule: rule,
CipherInfo: cipher,
StorageCacheId: importer.cacheKey,
}
log.Debug("download SST",
logutil.SSTMeta(&sstMeta),
Expand Down Expand Up @@ -715,6 +721,7 @@ func (importer *FileImporter) downloadRawKVSST(
RewriteRule: rule,
IsRawKv: true,
CipherInfo: cipher,
StorageCacheId: importer.cacheKey,
}
log.Debug("download SST", logutil.SSTMeta(&sstMeta), logutil.Region(regionInfo.Region))

Expand Down