Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support calculating historical timestamp #21522

Closed
Yisaer opened this issue Dec 7, 2020 · 3 comments
Closed

Support calculating historical timestamp #21522

Yisaer opened this issue Dec 7, 2020 · 3 comments
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@Yisaer
Copy link
Contributor

Yisaer commented Dec 7, 2020

Development Task

In #19817 , we are going to support timestamp bounded read-only transactions. There are 2 kinds of query for the timestamp bounded read-only transactions

  1. exact staleness
-- start a read-only transaction which reads the exact snapshot 5 seconds ago:
START TRANSACTION READ ONLY WITH TIMESTAMP BOUND EXACT STALENESS '00:00:05';

-- start a read-only transaction which reads the exact snapshot at "2020-09-06 00:00:00":
START TRANSACTION READ ONLY WITH TIMESTAMP BOUND READ TIMESTAMP '2020-09-06 00:00:00';

For the first query, we need to generate a TSO which represents for the TSO 5 secs ago. For the second query, we need to generate a TSO which represents the timestamp 2020-09-06 00:00:00. After the TSO generated, the tidb-server will send the request by holding the stale TSO to read the stale data.

  1. time-bounded staleness
-- start a read-only transaction which reads the data no stale than 10 seconds ago:
START TRANSACTION READ ONLY WITH TIMESTAMP BOUND MAX STALENESS '00:00:10'

-- start a read-only transaction which reads the data no stale than '2019-11-04 00:00:00':
START TRANSACTION READ ONLY WITH TIMESTAMP BOUND MIN READ TIMESTAMP '2019-11-04 00:00:00';

For the first query, we need to generate a TSO which represents for the TSO 10 secs ago. For the second query, we need to generate a TSO which represents the timestamp 2019-11-04 00:00:00. After the TSO generated, the tidb-server will send the request hold holding the TSO range [stale TSO, latest TSO] to read the time-bounded data.

Implementation

The tidb-server use Oracle interface to request TSO from PD. The TSO including 2 part, the physical part and the logical part. The accuracy of the physical part is ms.

For the stale TSO like 10 sec ago, we will provide a function in Oraclie like GetStaleTimestamp(prev int) and prev indicates for the seconds. To achieve this target, we will split this task in 2 sides.

PD

In PD, we need to mantain the last GlobalTSO and its arrival timestamp in pdClient. The pdClient should provide a function like GetLastGlobalTSO()(currentyly we only support global)

TiDB

In TiDB, GetStaleTimestamp(prev int) will directly call the GetLastGlobalTSO() and calculate the stale TSO. For example, the 10 secs ago stale TSO would be generated like following:

 last global TSO - (tidb_clock_time(lastTS arrived) - tidb_clock_time(10s ago))

As each tidb-server will call Global TSO each 2 seconds, the difference inaccuracy error will be limited into 2 sec which is accptable.

@Yisaer Yisaer added the type/enhancement The issue or PR belongs to an enhancement. label Dec 7, 2020
@Yisaer

This comment has been minimized.

@Yisaer
Copy link
Contributor Author

Yisaer commented Dec 8, 2020

/assign @Andrewmatilde

@djshow832
Copy link
Contributor

djshow832 commented Dec 23, 2020

For the stale TSO like 10 sec ago, we will provide a function in Oracle like GetStaleTimestamp(prev int) and prev indicates for the seconds.

What about:

START TRANSACTION READ ONLY WITH TIMESTAMP BOUND EXACT STALENESS '00:00:00.500';

I suggest supporting a more granular timestamp in case it's needed in the future.

@nolouch nolouch closed this as completed Sep 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants