Introduce cluster default remote translog buffer interval setting #9584

ashking94 · 2023-08-28T17:01:01Z

Description

This PR aims to introduce notion of cluster level default remote translog buffer interval setting. More details are present in #9574

Related Issues

Resolves #9574

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-08-28T17:11:28Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23630/
CommitID: 1158821
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-28T17:38:15Z

Compatibility status:

Checks if related components are compatible with change 1158821

Incompatible components

Incompatible components: [https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-28T18:05:56Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23633/
CommitID: 2597584
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-31T08:23:04Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards

URL: https://build.ci.opensearch.org/job/gradle-check/23934/
CommitID: 6f16d25
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2023-08-31T08:28:04Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23935/
CommitID: 5de80dc
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-31T09:06:25Z

Compatibility status:

Checks if related components are compatible with change 4d6849d

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-31T09:22:05Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/23942/
CommitID: 4d6849d

Signed-off-by: Ashish Singh <[email protected]>

github-actions · 2023-08-31T15:07:46Z

Compatibility status:

Checks if related components are compatible with change cfc4bb8

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-31T15:15:17Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/23979/
CommitID: cfc4bb8
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-08-31T15:17:33Z

Compatibility status:

Checks if related components are compatible with change 015257d

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-08-31T15:21:30Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/23980/
CommitID: 015257d

Signed-off-by: Ashish <[email protected]>

github-actions · 2023-09-01T07:21:40Z

Compatibility status:

Checks if related components are compatible with change fd8e526

Incompatible components

Incompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

github-actions · 2023-09-01T07:36:42Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/24079/
CommitID: fd8e526
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

ashking94 · 2023-09-01T09:04:31Z

Flaky test - #6445

Signed-off-by: Ashish Singh <[email protected]>

opensearch-trigger-bot · 2023-09-01T11:57:19Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9584-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 f5d3fd2483f58cdbb0d77cf53a1e5439b3cc860a
# Push it to GitHub
git push --set-upstream origin backport/backport-9584-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-9584-to-2.x.

github-actions · 2023-09-01T12:27:12Z

Compatibility status:

Checks if related components are compatible with change 9eb845e

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

github-actions · 2023-09-01T12:34:19Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/24099/
CommitID: 9eb845e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]>

ashking94 · 2023-09-04T06:55:17Z

Created manual backport #9723 since automated one failed.

) (#9723) Signed-off-by: Ashish Singh <[email protected]>

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Kaushal Kumar <[email protected]>

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Ivan Brusic <[email protected]>

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

ashking94 force-pushed the 9574 branch from 1158821 to 2597584 Compare August 28, 2023 17:22

ashking94 force-pushed the 9574 branch from 2597584 to 212b7f3 Compare August 28, 2023 20:07

ashking94 marked this pull request as ready for review August 28, 2023 20:09

Incorporate PR review comments

cfc4bb8

Signed-off-by: Ashish Singh <[email protected]>

ashking94 force-pushed the 9574 branch from 4d6849d to cfc4bb8 Compare August 31, 2023 14:29

Merge remote-tracking branch 'upstream/main' into 9574

015257d

Signed-off-by: Ashish Singh <[email protected]>

gbbafna approved these changes Sep 1, 2023

View reviewed changes

Merge branch 'main' into 9574

fd8e526

Signed-off-by: Ashish <[email protected]>

Empty-Commit

9eb845e

Signed-off-by: Ashish Singh <[email protected]>

gbbafna merged commit f5d3fd2 into opensearch-project:main Sep 1, 2023
8 of 9 checks passed

gbbafna added the backport 2.x Backport to 2.x branch label Sep 1, 2023

opensearch-trigger-bot bot added the backport-failed label Sep 1, 2023

ashking94 added a commit to ashking94/OpenSearch that referenced this pull request Sep 4, 2023

Introduce cluster default remote translog buffer interval setting (op…

7aaab87

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]>

ashking94 mentioned this pull request Sep 4, 2023

[Backport 2.x] Introduce cluster default remote translog buffer interval setting (#9584) #9723

Merged

sachinpkale pushed a commit that referenced this pull request Sep 4, 2023

Introduce cluster default remote translog buffer interval setting (#9584

1c6bc4e

) (#9723) Signed-off-by: Ashish Singh <[email protected]>

brusic pushed a commit to brusic/OpenSearch that referenced this pull request Sep 25, 2023

Introduce cluster default remote translog buffer interval setting (op…

30cf506

…ensearch-project#9584) Signed-off-by: Ashish Singh <[email protected]> Signed-off-by: Ivan Brusic <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce cluster default remote translog buffer interval setting #9584

Introduce cluster default remote translog buffer interval setting #9584

ashking94 commented Aug 28, 2023 •

edited

Loading

github-actions bot commented Aug 28, 2023

github-actions bot commented Aug 28, 2023

github-actions bot commented Aug 28, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Aug 31, 2023

github-actions bot commented Sep 1, 2023

github-actions bot commented Sep 1, 2023

ashking94 commented Sep 1, 2023

opensearch-trigger-bot bot commented Sep 1, 2023

github-actions bot commented Sep 1, 2023

github-actions bot commented Sep 1, 2023

ashking94 commented Sep 4, 2023

Introduce cluster default remote translog buffer interval setting #9584

Introduce cluster default remote translog buffer interval setting #9584

Conversation

ashking94 commented Aug 28, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Aug 28, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 28, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 28, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 31, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 31, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 31, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 31, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 31, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 31, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Aug 31, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Aug 31, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 1, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 1, 2023

Gradle Check (Jenkins) Run Completed with:

ashking94 commented Sep 1, 2023

opensearch-trigger-bot bot commented Sep 1, 2023

github-actions bot commented Sep 1, 2023

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 1, 2023

Gradle Check (Jenkins) Run Completed with:

ashking94 commented Sep 4, 2023

ashking94 commented Aug 28, 2023 •

edited

Loading