Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce KeySpace interface for API v2. #353

Open
wants to merge 46 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
1f314ee
fix type checking for new interface
iosmanthus Jun 21, 2022
aa6e4b5
cargo fmt
iosmanthus Jun 21, 2022
222908e
git fire: changing sig for PdRpcClient
iosmanthus Jun 22, 2022
605aed8
fix TxnApiV1 test
iosmanthus Jun 22, 2022
9164cf0
fix all tests for API v1
iosmanthus Jun 23, 2022
7668c37
fix integration tests for API v1
iosmanthus Jun 23, 2022
dd1c7b9
remove specification feature
iosmanthus Jun 23, 2022
60585f6
resolve conflict from master
iosmanthus Jun 24, 2022
3a0df00
reduce type annotation
iosmanthus Jun 24, 2022
7c3de1a
recover impl Trait for resolve_locks*
iosmanthus Jun 24, 2022
02377b7
git fire!: wip api v2
iosmanthus Jun 28, 2022
59002d1
impl KeyspaceCodec, along with RawKeyspaceCodec and TxnKeyspaceCodec
iosmanthus Jul 1, 2022
a440794
fix fmt check
iosmanthus Jul 1, 2022
0a72d31
rename request_codec.rs to codec.rs
iosmanthus Jul 1, 2022
19c3123
git fire!: wip impl RequestCodec for requests
iosmanthus Jul 1, 2022
624c68e
impl KvRequest for raw requests
iosmanthus Jul 4, 2022
71af50d
cargo fmt
iosmanthus Jul 5, 2022
a5f326f
impl KvRequest for txn requests
iosmanthus Jul 5, 2022
5efbc12
make check
iosmanthus Jul 5, 2022
b534cfd
impl Default for KeyspaceId
iosmanthus Jul 5, 2022
434e10d
cargo fmt
iosmanthus Jul 6, 2022
317116b
decode in place
iosmanthus Jul 6, 2022
5c5809a
introduce impl_kv_request to simplify code
iosmanthus Jul 6, 2022
701964d
refactor impl_kv_request macro
iosmanthus Jul 6, 2022
db38d95
fix tests
iosmanthus Jul 6, 2022
3660ca1
move default impl into RequestCodecExt
iosmanthus Jul 7, 2022
7d68247
fix api version context
iosmanthus Jul 7, 2022
81cbddd
move request codec into raw/txn owned module
iosmanthus Jul 8, 2022
5bd0f6e
fix doc test
iosmanthus Jul 8, 2022
1c1ce17
fix wrong TxnCodec mark trait for raw::Keyspace
iosmanthus Jul 9, 2022
051a157
refactor ApiVx with Mode generic para
iosmanthus Jul 11, 2022
159de5c
Merge branch 'master' of github.com:tikv/client-rust into api-v2
iosmanthus Jul 11, 2022
bfd1f28
remove dynamic dispatch while encoding request in Dispatch
iosmanthus Jul 11, 2022
7fc61ea
make IsDefault private in crate
iosmanthus Jul 11, 2022
6baaa94
remove impl Deref for Key
iosmanthus Jul 11, 2022
07c7af7
fix potential index out of range while meets corrupted keyspace prefix
iosmanthus Jul 12, 2022
0ed902d
introduce codec type parameter for Snapshot and Transcation
iosmanthus Jul 12, 2022
182a457
bound Client to right codec
iosmanthus Jul 12, 2022
0e76e10
fix encode range issue with unbound end
iosmanthus Jul 14, 2022
0c3c9e3
Merge branch 'master' of github.com:tikv/client-rust into api-v2
iosmanthus Jul 14, 2022
f46cea8
introduce newer version of proto
iosmanthus Jul 21, 2022
130e1b0
git fire: impl load keyspace
iosmanthus Jul 21, 2022
3819c8a
introduce keyspace mgr from keyspace service
iosmanthus Jul 21, 2022
a2047eb
fix clippy
iosmanthus Jul 22, 2022
45af3c5
refine trait bound of raw::Client
iosmanthus Jul 22, 2022
9de6f50
remove phatom of txn::Client
iosmanthus Jul 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions examples/pessimistic.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
mod common;

use crate::common::parse_args;
use tikv_client::{Config, Key, TransactionClient as Client, TransactionOptions, Value};
use tikv_client::{
request::request_codec::TxnApiV1, Config, Key, TransactionClient as Client, TransactionOptions,
Value,
};

#[tokio::main]
async fn main() {
Expand All @@ -20,7 +23,7 @@ async fn main() {
};

// init
let client = Client::new_with_config(args.pd, config, None)
let client = Client::new_with_config(args.pd, config, TxnApiV1, None)
Copy link
Member

@sunxiaoguang sunxiaoguang Jun 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client itself can tell we are using TxnKV already, can we hide things like TxnApiV1, TxnApiV2, RawApiV1, RawApiV2. Just let user chose the appropriate client and use ApiV1 or ApiV2.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may reexport them in their own module, like: raw::ApiV1 , txn::ApiV1

.await
.expect("Could not connect to tikv");

Expand Down
8 changes: 5 additions & 3 deletions examples/raw.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
mod common;

use crate::common::parse_args;
use tikv_client::{Config, IntoOwnedRange, Key, KvPair, RawClient as Client, Result, Value};
use tikv_client::{
request::request_codec::RawApiV1, Config, IntoOwnedRange, Key, KvPair, RawClient as Client,
Result, Value,
};

const KEY: &str = "TiKV";
const VALUE: &str = "Rust";
Expand All @@ -26,8 +29,7 @@ async fn main() -> Result<()> {

// When we first create a client we receive a `Connect` structure which must be resolved before
// the client is actually connected and usable.
let client = Client::new_with_config(args.pd, config, None).await?;
let client = client.clone();
let client = Client::new_with_config(args.pd, config, RawApiV1, None).await?;

// Requests are created from the connected client. These calls return structures which
// implement `Future`. This means the `Future` must be resolved before the action ever takes
Expand Down
17 changes: 10 additions & 7 deletions examples/transaction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
mod common;

use crate::common::parse_args;
use tikv_client::{BoundRange, Config, Key, KvPair, TransactionClient as Client, Value};
use tikv_client::{
request::request_codec::{TxnApiV1, TxnCodec},
BoundRange, Config, Key, KvPair, TransactionClient as Client, Value,
};

async fn puts(client: &Client, pairs: impl IntoIterator<Item = impl Into<KvPair>>) {
async fn puts<C: TxnCodec>(client: &Client<C>, pairs: impl IntoIterator<Item = impl Into<KvPair>>) {
let mut txn = client
.begin_optimistic()
.await
Expand All @@ -17,7 +20,7 @@ async fn puts(client: &Client, pairs: impl IntoIterator<Item = impl Into<KvPair>
txn.commit().await.expect("Could not commit transaction");
}

async fn get(client: &Client, key: Key) -> Option<Value> {
async fn get<C: TxnCodec>(client: &Client<C>, key: Key) -> Option<Value> {
let mut txn = client
.begin_optimistic()
.await
Expand All @@ -29,7 +32,7 @@ async fn get(client: &Client, key: Key) -> Option<Value> {
res
}

async fn key_exists(client: &Client, key: Key) -> bool {
async fn key_exists<C: TxnCodec>(client: &Client<C>, key: Key) -> bool {
let mut txn = client
.begin_optimistic()
.await
Expand All @@ -44,7 +47,7 @@ async fn key_exists(client: &Client, key: Key) -> bool {
res
}

async fn scan(client: &Client, range: impl Into<BoundRange>, limit: u32) {
async fn scan<C: TxnCodec>(client: &Client<C>, range: impl Into<BoundRange>, limit: u32) {
let mut txn = client
.begin_optimistic()
.await
Expand All @@ -56,7 +59,7 @@ async fn scan(client: &Client, range: impl Into<BoundRange>, limit: u32) {
txn.commit().await.expect("Could not commit transaction");
}

async fn dels(client: &Client, keys: impl IntoIterator<Item = Key>) {
async fn dels<C: TxnCodec>(client: &Client<C>, keys: impl IntoIterator<Item = Key>) {
let mut txn = client
.begin_optimistic()
.await
Expand All @@ -81,7 +84,7 @@ async fn main() {
Config::default()
};

let txn = Client::new_with_config(args.pd, config, None)
let txn = Client::new_with_config(args.pd, config, TxnApiV1, None)
.await
.expect("Could not connect to tikv");

Expand Down
12 changes: 8 additions & 4 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,9 @@
//! # use tikv_client::{RawClient, Result};
//! # use futures::prelude::*;
//! # fn main() -> Result<()> {
//! # futures::executor::block_on(async {
//! let client = RawClient::new(vec!["127.0.0.1:2379"], None).await?;
//! # use tikv_client::request::request_codec::RawApiV1;
//! futures::executor::block_on(async {
//! let client = RawClient::new(vec!["127.0.0.1:2379"], RawApiV1, None).await?;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might break the original API, we should add a default implementation for API V1.

//! client.put("key".to_owned(), "value".to_owned()).await?;
//! let value = client.get("key".to_owned()).await?;
//! # Ok(())
Expand All @@ -80,8 +81,9 @@
//! # use tikv_client::{TransactionClient, Result};
//! # use futures::prelude::*;
//! # fn main() -> Result<()> {
//! # futures::executor::block_on(async {
//! let txn_client = TransactionClient::new(vec!["127.0.0.1:2379"], None).await?;
//! # use tikv_client::request::request_codec::TxnApiV1;
//! futures::executor::block_on(async {
//! let txn_client = TransactionClient::new(vec!["127.0.0.1:2379"], TxnApiV1, None).await?;
//! let mut txn = txn_client.begin_optimistic().await?;
//! txn.put("key".to_owned(), "value".to_owned()).await?;
//! let value = txn.get("key".to_owned()).await?;
Expand Down Expand Up @@ -124,6 +126,8 @@ pub use crate::backoff::Backoff;
#[doc(inline)]
pub use crate::kv::{BoundRange, IntoOwnedRange, Key, KvPair, Value};
#[doc(inline)]
pub use crate::pd::PdClient;
#[doc(inline)]
pub use crate::raw::{lowering as raw_lowering, Client as RawClient, ColumnFamily};
#[doc(inline)]
pub use crate::request::RetryOptions;
Expand Down
10 changes: 8 additions & 2 deletions src/mock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
use crate::{
pd::{PdClient, PdRpcClient, RetryClient},
region::{RegionId, RegionWithLeader},
request::request_codec::RawApiV1,
store::RegionStore,
Config, Error, Key, Result, Timestamp,
};
Expand All @@ -20,7 +21,7 @@ use tikv_client_store::{KvClient, KvConnect, Request};

/// Create a `PdRpcClient` with it's internals replaced with mocks so that the
/// client can be tested without doing any RPC calls.
pub async fn pd_rpc_client() -> PdRpcClient<MockKvConnect, MockCluster> {
pub async fn pd_rpc_client() -> PdRpcClient<RawApiV1, MockKvConnect, MockCluster> {
let config = Config::default();
let plain = slog_term::PlainSyncDecorator::new(std::io::stdout());
let logger = Logger::root(
Expand All @@ -41,7 +42,7 @@ pub async fn pd_rpc_client() -> PdRpcClient<MockKvConnect, MockCluster> {
MockCluster,
))
},
false,
RawApiV1,
logger,
)
.await
Expand Down Expand Up @@ -153,6 +154,7 @@ impl MockPdClient {
#[async_trait]
impl PdClient for MockPdClient {
type KvClient = MockKvClient;
type RequestCodec = RawApiV1;

async fn map_region_to_store(self: Arc<Self>, region: RegionWithLeader) -> Result<RegionStore> {
Ok(RegionStore::new(region, Arc::new(self.client.clone())))
Expand Down Expand Up @@ -197,4 +199,8 @@ impl PdClient for MockPdClient {
}

async fn invalidate_region_cache(&self, _ver_id: crate::region::RegionVerId) {}

fn get_request_codec(&self) -> Self::RequestCodec {
RawApiV1
}
}
Loading