Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[META] [Segment Replication] Integration with remote store #4448

Closed
7 of 8 tasks
Tracked by #2194
dreamer-89 opened this issue Sep 7, 2022 · 1 comment
Closed
7 of 8 tasks
Tracked by #2194

[META] [Segment Replication] Integration with remote store #4448

dreamer-89 opened this issue Sep 7, 2022 · 1 comment
Labels
discuss Issues intended to help drive brainstorming and decision making distributed framework enhancement Enhancement or improvement to existing feature or request Indexing:Replication Issues and PRs related to core replication framework eg segrep v2.10.0

Comments

@dreamer-89
Copy link
Member

dreamer-89 commented Sep 7, 2022 •
edited by mch2
Loading

With remote store integration, the segments files for index shards will be stored on remote store. Replica shards will copy these files from remote store thus reducing the I/O on primary shard. As part of this issue, integration with remote translog is not handled. Thus, primary will continue to hold the translog ops on local disk.

Enhancements:

  • [Segment Replication + Remote Store] POC - Create POC for commit without fsync and rely on remote store + translog for durability.

Design

Remote Store #2700
Segment Replication #2229
@dreamer-89 dreamer-89 added enhancement Enhancement or improvement to existing feature or request untriaged v3.0.0 Issues and PRs related to version 3.0.0 labels Sep 7, 2022
@mch2 mch2 mentioned this issue Sep 7, 2022
Closed
31 tasks
@dreamer-89 dreamer-89 added discuss Issues intended to help drive brainstorming and decision making distributed framework and removed untriaged labels Sep 9, 2022
@anasalkouz anasalkouz changed the title [Segment Replication] Integration with remote store [META][Segment Replication] Integration with remote store Jun 8, 2023
@anasalkouz anasalkouz changed the title [META][Segment Replication] Integration with remote store [META] [Segment Replication] Integration with remote store Jun 8, 2023
@anasalkouz anasalkouz added v2.9.0 'Issues and PRs related to version v2.9.0' and removed v3.0.0 Issues and PRs related to version 3.0.0 labels Jun 15, 2023
@mch2 mch2 mentioned this issue Jun 16, 2023
Closed
@mch2 mch2 mentioned this issue Jun 27, 2023
Closed
4 tasks
@anasalkouz anasalkouz added v2.10.0 and removed v2.9.0 'Issues and PRs related to version v2.9.0' labels Jul 6, 2023
@tlfeng tlfeng mentioned this issue Jul 25, 2023
Closed
@Bukhtawar Bukhtawar added the Indexing:Replication Issues and PRs related to core replication framework eg segrep label Jul 27, 2023
@mch2
Copy link
Member

mch2 commented Aug 23, 2023

Closing this issue, remaining tasks are tracked in https://github.com/orgs/opensearch-project/projects/99

@mch2 mch2 closed this as completed Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discuss Issues intended to help drive brainstorming and decision making distributed framework enhancement Enhancement or improvement to existing feature or request Indexing:Replication Issues and PRs related to core replication framework eg segrep v2.10.0
Projects
Segment Replication
Status: Done
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mch2 @Bukhtawar @anasalkouz @dreamer-89