Remove wal #1

5kbpers · 2022-09-23T02:20:00Z

What is changed and how it works?

Issue Number: Close #xxx

What's Changed:

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
Need to cherry-pick to the release branch

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression
- Consumes more CPU
- Consumes more MEM
Breaking backward compatibility

Release note

Please add a release note.

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with None.
If this PR will be picked to release branch, then a release note is probably required.

Signed-off-by: 5kbpers <[email protected]>

ref tikv#12842 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

ref tikv#13196 Now, all integration test cases uses checkpoint V3. Removed `test_inflight_message` because it is invalid in V3. Signed-off-by: Yu Juncen <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

ref tikv#12876 Signed-off-by: Wenbo Zhang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

…k state. (tikv#13541) ref tikv#13303, ref tikv#13519 Add a Raft admin command to put the region into a lock state to prevent any reading, writing, and scheduling and persist the state in the RegionLocalState. Signed-off-by: husharp <[email protected]>

close tikv#13394 Otherwise it will waste CPU on loop. Signed-off-by: Jay Lee <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

…ue (tikv#13584) ref tikv#13298 Avoid stale entries in the new lock waiting queue. This is done by making use of another implementation of the priority queue (instead of the std BinaryHeap) that supports efficiently removing element by key. So that when a lock-waiting request is canceled, the entry can be removed from the queue immediately, instead of waiting for the lazy-cleaning-up. Signed-off-by: MyonKeminta <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]> Co-authored-by: Yilin Chen <[email protected]>

close tikv#13468 Signed-off-by: lizhenhuan <[email protected]> Signed-off-by: 3pointer <[email protected]> Signed-off-by: Leavrth <[email protected]> Signed-off-by: CalvinNeo <[email protected]> Co-authored-by: 3pointer <[email protected]> Co-authored-by: Jianjun Liao <[email protected]> Co-authored-by: Calvin Neo <[email protected]> Co-authored-by: Yilin Chen <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

…ikv#13568) close tikv#13553 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

…13597) ref tikv#13596, close tikv#13596 Rename `causal-ts.available-interval` to `causal-ts.alloc-ahead-buffer` for more clear meaning. Signed-off-by: pingyu <[email protected]>

close tikv#13553 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

close tikv#13519, ref tikv#13519, ref tikv#13541 Make `FlashbackToVersion` become a two-phase request as described in tikv#13519. Signed-off-by: JmPotato <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

…kv#13618) ref tikv#13303, ref pingcap/tidb#38475 Move `check_flashback_state` to after `check_region_epoch` to make sure the Region Cache on the client-side could be refreshed ASAP. Signed-off-by: JmPotato <[email protected]>

close tikv#13599 The logic in `test_raw_put_key_guard` is incorrect, fix it. Signed-off-by: haojinming <[email protected]> Co-authored-by: Ping Yu <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

ref tikv#13303 Refine the flashback raftstore test. Signed-off-by: JmPotato <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

close tikv#13353 Signed-off-by: hehechen <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

ref tikv#13131 None Signed-off-by: tabokie <[email protected]>

close tikv#13625 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

Signed-off-by: 5kbpers <[email protected]>

5kbpers added 30 commits July 7, 2022 17:05

update rocksdb

46b9a80

Signed-off-by: 5kbpers <[email protected]>

return seq in apply res

61c4c03

Signed-off-by: 5kbpers <[email protected]>

sync sequence number from apply to store

d7d3afe

Signed-off-by: 5kbpers <[email protected]>

persist relations

3eff1f8

Signed-off-by: 5kbpers <[email protected]>

move sequence number to tikv_util

77a1d62

Signed-off-by: 5kbpers <[email protected]>

add flush listener

f1796d2

Signed-off-by: 5kbpers <[email protected]>

update kvproto

975e58b

Signed-off-by: 5kbpers <[email protected]>

add option

347eeb5

Signed-off-by: 5kbpers <[email protected]>

add test for sequence number window

487458f

Signed-off-by: 5kbpers <[email protected]>

add flush listener if kv wal disabled

432fe3f

Signed-off-by: 5kbpers <[email protected]>

sync region local state

be9ee76

Signed-off-by: 5kbpers <[email protected]>

add seqno properties

c3ddcac

Signed-off-by: 5kbpers <[email protected]>

update raft_engine

2dd5179

Signed-off-by: 5kbpers <[email protected]>

recover from raftdb

8dc5ef3

Signed-off-by: 5kbpers <[email protected]>

add memtable sealed counter

a846c3f

Signed-off-by: 5kbpers <[email protected]>

delay store relations to raft engine

b34cfb5

Signed-off-by: 5kbpers <[email protected]>

add test

694d216

Signed-off-by: 5kbpers <[email protected]>

Merge remote-tracking branch 'upstream/master' into remove-wal

650f66a

Signed-off-by: 5kbpers <[email protected]>

Merge remote-tracking branch 'upstream/master' into remove-wal

ec645fb

Signed-off-by: 5kbpers <[email protected]>

update rocksdb

c841d3b

Signed-off-by: 5kbpers <[email protected]>

comment init

f71279a

Signed-off-by: 5kbpers <[email protected]>

remove push seqno

91f5623

Signed-off-by: 5kbpers <[email protected]>

unwrap

35f2e2e

Signed-off-by: 5kbpers <[email protected]>

on exit

518fb82

Signed-off-by: 5kbpers <[email protected]>

add seqno worker

a850f14

Signed-off-by: 5kbpers <[email protected]>

use one cell

ed8dc63

Signed-off-by: 5kbpers <[email protected]>

init raftlog gc scheduler

4f400bd

Signed-off-by: 5kbpers <[email protected]>

seqno relations metrics

4e05e34

Signed-off-by: 5kbpers <[email protected]>

split raft wb

99270c8

Signed-off-by: 5kbpers <[email protected]>

fix panic

218a001

Signed-off-by: 5kbpers <[email protected]>

5kbpers force-pushed the remove-wal branch from 1cd5303 to c2dfb33 Compare September 24, 2022 09:43

meta deadlock

e463baa

Signed-off-by: 5kbpers <[email protected]>

5kbpers force-pushed the remove-wal branch from c5644e3 to e463baa Compare September 24, 2022 16:00

5kbpers and others added 14 commits September 25, 2022 11:59

handle commit merge

99fcfd2

Signed-off-by: 5kbpers <[email protected]>

delay cleanup

586ddd7

Signed-off-by: 5kbpers <[email protected]>

Merge remote-tracking branch 'upstream/master' into remove-wal

732002e

Signed-off-by: 5kbpers <[email protected]>

fix seqno commit

44aee45

Signed-off-by: 5kbpers <[email protected]>

fix delayed destroy

3bb4dbe

Signed-off-by: 5kbpers <[email protected]>

Merge remote-tracking branch 'upstream/master' into remove-wal

f5875a2

Signed-off-by: 5kbpers <[email protected]>

raftstore-v2: implement local read for raftstore-v2 (tikv#13375)

94c8aa5

ref tikv#12842 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

log-backup: remove checkpoint V2 from codebase (tikv#13197)

be44dba

ref tikv#13196 Now, all integration test cases uses checkpoint V3. Removed `test_inflight_message` because it is invalid in V3. Signed-off-by: Yu Juncen <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

trace peers' availability info on leader side (tikv#13209)

f702db2

ref tikv#12876 Signed-off-by: Wenbo Zhang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

channel: early break when fetch nothing (tikv#13516)

066a422

close tikv#13394 Otherwise it will waste CPU on loop. Signed-off-by: Jay Lee <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

local_reader: solve the race condition when acquiring the snapshot (t…

11a340c

…ikv#13568) close tikv#13553 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

5kbpers force-pushed the remove-wal branch from 2aa81c1 to 32200fe Compare October 13, 2022 08:36

pingyu and others added 10 commits October 14, 2022 12:59

causal-ts: rename available-interval to alloc-ahead-buffer (tikv#…

b448214

…13597) ref tikv#13596, close tikv#13596 Rename `causal-ts.available-interval` to `causal-ts.alloc-ahead-buffer` for more clear meaning. Signed-off-by: pingyu <[email protected]>

local_reader: release snapshot properly (tikv#13605)

26b6c3c

close tikv#13553 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

*: introduce the two-phase kv_flashback_to_version (tikv#13557)

b37c2f6

close tikv#13519, ref tikv#13519, ref tikv#13541 Make `FlashbackToVersion` become a two-phase request as described in tikv#13519. Signed-off-by: JmPotato <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

test: Fix incorrect rawkv case test_raw_put_key_guard (tikv#13600)

571b5a2

close tikv#13599 The logic in `test_raw_put_key_guard` is incorrect, fix it. Signed-off-by: haojinming <[email protected]> Co-authored-by: Ping Yu <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

tests: refine the flashback raftstore test (tikv#13615)

13f58a9

ref tikv#13303 Refine the flashback raftstore test. Signed-off-by: JmPotato <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

resolved_ts: track 1PC (tikv#13579)

ed64ed2

close tikv#13353 Signed-off-by: hehechen <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

*: fix build with panic engines (tikv#13585)

a63944c

ref tikv#13131 None Signed-off-by: tabokie <[email protected]>

raftstore-v2: test_read is not stable (tikv#13626)

08b5a4a

close tikv#13625 Signed-off-by: SpadeA-Tang <[email protected]> Co-authored-by: Ti Chi Robot <[email protected]>

refactor

692ba50

Signed-off-by: 5kbpers <[email protected]>

5kbpers force-pushed the remove-wal branch from a019f42 to 692ba50 Compare October 19, 2022 09:17

Merge remote-tracking branch 'upstream/master' into remove-wal

47d8365

Signed-off-by: 5kbpers <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove wal #1

Remove wal #1

5kbpers commented Sep 23, 2022

Remove wal #1

Are you sure you want to change the base?

Remove wal #1

Conversation

5kbpers commented Sep 23, 2022

What is changed and how it works?

Related changes

Check List

Release note