Skip to content

Commit

Permalink
Merge pull request pingcap#1 from iosmanthus/iosmanthus/keyspace-rfc
Browse files Browse the repository at this point in the history
Add client-go and coprocessor section
  • Loading branch information
ystaticy authored Dec 12, 2022
2 parents 862e07d + 634a9fc commit 16d9467
Showing 1 changed file with 45 additions and 2 deletions.
47 changes: 45 additions & 2 deletions docs/design/2022-12-06-Keyspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,52 @@ To be supplemented in another PR

### TiKV
#### client-go
To be supplemented in another PR

client-go should also support API V2 and Keyspace, which will make all the codec logic transparent to upper users. We currently maintain these implementations in [tikv/client-go/api-v2](https://github.com/tikv/client-go/tree/api-v2).

The new version of client-go unifies all the codec logic by introducing a new interface call `Codec`.
```go
// Codec is responsible for encode/decode requests.
type Codec interface {
// ...
EncodeRequest(req *tikvrpc.Request) (*tikvrpc.Request, error)
// DecodeResponse decode the resp with the given codec.
DecodeResponse(req *tikvrpc.Request, resp *tikvrpc.Response) (*tikvrpc.Response, error)
// EncodeRegionKey encode region's key.
EncodeRegionKey(key []byte) []byte
// DecodeRegionKey decode region's key
DecodeRegionKey(encodedKey []byte) ([]byte, error)
// EncodeRegionRange encode region's start and end.
EncodeRegionRange(start, end []byte) ([]byte, []byte)
// DecodeRegionRange decode region's start and end.
DecodeRegionRange(encodedStart, encodedEnd []byte) ([]byte, []byte, error)
// EncodeRange encode a key range.
EncodeRange(start, end []byte) ([]byte, []byte)
// DecodeRange decode a key range.
DecodeRange(encodedStart, encodedEnd []byte) ([]byte, []byte, error)
// EncodeKey encode a key.
EncodeKey(key []byte) []byte
// DecodeKey decode a key.
DecodeKey(encoded []byte) ([]byte, error)
}
```

This interface will encode the key-related request right before it is sent to the TiKV/TiFlash/PD server, and decode the response right after it is received. This design should neat enough to make encoding and decoding transparent to the upper users.

Typically, [`codecV2`](https://github.com/tikv/client-go/blob/239ac1b2b7fc67921b00e1d51d47f3716c2c2f0c/internal/apicodec/codec_v2.go#L41) implements the `Codec` interface and all the Keyspace related codec logic.
The encode logic is trivial, but the decode logic is a little bit complicated.
To achive transparency, `codecV2` will map the region range into `[0, +inf)`, if the range is not overlapped with the current keyspace, it will return an error.
For example, if the keyspace is `[x001, x002)`, then the region range `[x003, x004)` is invalid, and `[x000, x002)` is mapped to `[0, +inf)`.

client-go provides a function called `NewCodecPDClientWithKeyspace` to create a PD client with Keyspace support. It try to fetch the keyspace id with the given keyspace name from PD, and create a `codecV2` with the keyspace id.
You could get the `Codec` within the `CodecPDClient` and use it to construct a "codeced" TiKV gRPC client.

#### Coprocessor
To be supplemented in another PR

TiKV Coprocessor use the range in the request to determine which range it should scan.
After scanning, it will transform the key/value pair into the columnar format and begin to execute the DAG request.
However, the original code could not recognize the new format key.
So the [`RangeScanner`](https://github.com/iosmanthus/tikv/components/tidb_query_common/src/storage/scanner.rs#L20) should be modified to support the new format key by ignore the first 4 bytes if the request is using API V2.



0 comments on commit 16d9467

Please sign in to comment.