Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Add max_shard_size parameter to shrink API page #2352

Closed
wants to merge 2 commits into from

Conversation

gaobinlong
Copy link
Contributor

@gaobinlong gaobinlong commented Jan 9, 2023

Signed-off-by: gaobinlong [email protected]

Description

Relates to #2044.
The main change of this PR is adding a new parameter named max_shard_size to shrink API page, the parameter was added to OpenSearch core in this PR: 5229, so we need to add the parameter to the documentation.

Issues Resolved

Closes #2044

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@gaobinlong gaobinlong requested a review from a team as a code owner January 9, 2023 13:50
@gaobinlong gaobinlong changed the title Add max_shard_size parameter to shrink API page (#2044) [DOC] Add max_shard_size parameter to shrink API page Jan 9, 2023
@Naarcha-AWS Naarcha-AWS added backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 backport 2.4 PR: Backport label for 2.4 backport 2.5 PR: Backport label for 2.5 labels Jan 25, 2023
The new parameter to be called max_shard_size.
max_primary_shard_size | Bytes | Sets the maximum size of a primary shard in the target index. For example, if this field is set to 100 GB, and the source index's primary shards total to 300 GB, then the target index has 3 primary shards of 100 GB each.
{% endcomment %}
max_shard_size | Bytes | Sets the maximum size of a primary shard in the target index. `max_shard_size` calculates the target index's primary shard count and the primary shards' storage in the source index. The primary shard count of the target index is the lowest factor of the source index's primary shard count which satisfies this field's requirement. For example, if `max_shard_size` is set to `100 GB`, and the source index's primary shards total to `300 GB` and the count is `6`, then the target index has **3 primary shards of 100 GB each**. If the source index's primary shards total to `600GB` and count is `5`, then the target index will have **5 primary shards**. Because `max_shard_size` conflicts with `index.number_of_shards` in `settings` you can not set both of them at the same time.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaobinlong After looking at the code, I think we might want to break this parameter out in a separate section with more explanations and a non-trivial example. We might also want to mention the max and min for the number of primary shards in the target index. Do you mind if I take over this PR and try to do this, and you can review for technical accuracy?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kolchfa-aws sure, you can do it, and feel free to ping me if you need some help.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaobinlong Could you please review the new PR for technical accuracy? #2519

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure,I've added some comments for the new PR, you can take a look.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaobinlong Somehow I don't see any comments on #2519. I don't know if it's a problem on my end. Would you mind confirming that they are there?

@kolchfa-aws
Copy link
Collaborator

Closing this PR in favor of #2519

@kolchfa-aws kolchfa-aws closed this Feb 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 backport 2.4 PR: Backport label for 2.4 backport 2.5 PR: Backport label for 2.5
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Add max_shard_size parameter for Shrink API
3 participants