Skip to content
This repository has been archived by the owner on Jun 23, 2022. It is now read-only.

feat(bulk-load): meta server send bulk load request #457

Merged
merged 5 commits into from
May 13, 2020

Conversation

hycdong
Copy link
Contributor

@hycdong hycdong commented May 9, 2020

This pull request is about meta server send bulk_load_request to replica server.

  • meta server creates bulk_load_rpc and send bulk_load_request in function partition_bulk_load
  • primary replica calls on_bulk_load when receive bulk_load_request
    1. validate replica status and ballot
    2. execute actual bulk load process (do_bulk_load - not implemented in this pr )
    3. broadcast bulk load request to secondaries and report bulk load states to meta (broadcast_group_bulk_load, report_bulk_load_states_to_meta - not implemented in this pr)
  • meta server calls on_partition_bulk_load_reply when receive bulk_load_response, also not implemented in this pr.
    Whole bulk load process document can be found in official website: bulk_load_design

@hycdong hycdong changed the title feat(bulk-load): meta send bulk load request feat(bulk-load): meta server send bulk load request May 9, 2020
@hycdong hycdong marked this pull request as ready for review May 9, 2020 06:52
return;
}

broadcast_group_bulk_load(request);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whether broadcast_group_bulk_load success or not doesn't influence response?

Copy link
Contributor Author

@hycdong hycdong May 12, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • meta -> primary: send bulk _load_request
  • primary -> secondary: send group_bulk_load_request (through broadcast_group_bulk_load)
  • secondary -> primary : report bulk load state (in group_bulk_load_response)
  • primary -> meta: report group bulk load states (in bulk_load_response)

meta receives bulk_load_response, get group bulk load states, if bulk load is not finished, meta will resend bulk _load_request to primary.
If broadcast_group_bulk_load failed, the states in response reported to meta may not fresh. However, it is ok, meta will resend request and finally get the states. Besides, broadcasting group bulk load is not a synchronous process, the states in response will not always be fresh.

src/dist/replication/lib/replica_stub.cpp Outdated Show resolved Hide resolved
src/dist/replication/meta_server/meta_bulk_load_service.h Outdated Show resolved Hide resolved
src/dist/replication/meta_server/meta_bulk_load_service.h Outdated Show resolved Hide resolved
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants