-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Remote Store] Add capability of doing refresh as determined by the translog #12992
Merged
gbbafna
merged 18 commits into
opensearch-project:main
from
astute-decipher:refreshIntervalUpdate
Apr 25, 2024
Merged
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
f24757d
Adding support to allow refresh interval -1
5f4ca3a
Changing logic to Just trigger refresh, and leave trimtranslog on Aft…
0e1a2d3
Added IT/UT and changed setting scope
4fd9bf5
Minor correction
90bcd5e
Comments and CHANGELOG.md
cb0fc32
Make refresh non-blocking
f28ef5c
Added UT for IndexShard changes
c5df7d4
Some refactoring and name changes
29cb727
rebase from main
1718d64
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
d1ffa62
Code refactoring
d0c2dd2
Changed logic to do flush instead of refresh
6d93d6a
rebuild jenkins empty commit
efd3e17
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
312c58a
Addresed comment on PR
6330d16
Merge from main
8a81cb7
Addressed comment on PR
503f8ec
Merge branch 'main' of https://github.com/opensearch-project/OpenSear…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -437,6 +437,12 @@ | |
* @return if the translog should be flushed | ||
*/ | ||
public boolean shouldPeriodicallyFlush(long localCheckpointOfLastCommit, long flushThreshold) { | ||
/* | ||
* This can trigger flush depending upon translog's implementation | ||
*/ | ||
if (translog.shouldFlush()) { | ||
return true; | ||
} | ||
Comment on lines
+443
to
+445
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see reference of looping in the following java doc couple of lines below this change. Have we verified that there is no potential infinite loop or recursiveness here? |
||
// This is the minimum seqNo that is referred in translog and considered for calculating translog size | ||
long minTranslogRefSeqNo = translog.getMinUnreferencedSeqNoInSegments(localCheckpointOfLastCommit + 1); | ||
final long minReferencedTranslogGeneration = translog.getMinGenerationForSeqNo(minTranslogRefSeqNo).translogFileGeneration; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test will be increasing the test execution time to 90-100s. We can reduce the test execution by setting the translog buffer interval to zero. Can we please do this change?