Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat(split): parent replica prepare states #299

Merged
merged 22 commits into from
Sep 6, 2019

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented Aug 22, 2019

Simple partition split process

  1. meta receives client partition split request, and change partition count split: meta start partition split #286
  2. replica notices partition count changed during on_config_sync
  3. parent partition create child partition split: parent replica create child replica #291
  4. parent prepare states for child to learn
  5. child partition async learn mutations and private logs from parent
  6. all child partitions in group finish learn process, parent will stop read and write
  7. meta server register child partitions
  8. child partition active, and parent recover read and write

More partition split discussion in issue #69
This pr solves the fourth step of partition split, which is bold in process description.

What this pr solved

  • Parent prepare states for child to learn through function parent_prepare_states, states including:
    • checkpoint
    • private log files
    • mutations in memory
    • prepare list
  • Partition split is different from learn
    In learn. potential secondary replica learn states from primary replica on another replica server. However, child replica learn states from its parent on the same replica server and the same disk. As a result, parent replica generate checkpoint instead of sst files, child replica can read private log files directly rather than copy them, and there is no need to serialize mutations any more. Partition split is equivalent to combination of all learn staging (LT_APP, LT_LOG and LT_CACHE). Besides, child will copy parent's prepare_list in child_copy_states
  • Add new constructor of below classes, create a new instance by an old instance:
    • mutation
    • mutation_cache
    • prepare_list
    • message_ex

Tests

Add unit test for two new functions, parent_check_states, parent_prepare_states

@hycdong hycdong changed the title split: parent replica prepare states feat: partition split parent replica prepare states Aug 22, 2019
@hycdong hycdong changed the title feat: partition split parent replica prepare states feat: partition split - parent replica prepare states Aug 22, 2019
@neverchanje neverchanje changed the title feat: partition split - parent replica prepare states feat(split): parent replica prepare states Aug 23, 2019
src/dist/replication/lib/replica.h Show resolved Hide resolved
src/dist/replication/lib/replica_split.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/mutation_cache.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/mutation_log.h Outdated Show resolved Hide resolved
src/dist/replication/lib/prepare_list.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/mutation_log.h Outdated Show resolved Hide resolved
src/dist/replication/lib/mutation_cache.cpp Outdated Show resolved Hide resolved
include/dsn/tool-api/rpc_message.h Outdated Show resolved Hide resolved
src/dist/replication/lib/prepare_list.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/mutation_log.h Outdated Show resolved Hide resolved
neverchanje
neverchanje previously approved these changes Sep 5, 2019
src/dist/replication/lib/mutation.cpp Outdated Show resolved Hide resolved
src/dist/replication/lib/replica_split.cpp Outdated Show resolved Hide resolved
@neverchanje
Copy link
Contributor

我没有太大问题了,剩下几个 comments 改一下我觉得就可以过

neverchanje
neverchanje previously approved these changes Sep 6, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants