Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add deprecated API for creating History Ops Snapshot from translog #2886

Merged
merged 2 commits into from
Apr 15, 2022

Conversation

saikaranam-amazon
Copy link
Member

@saikaranam-amazon saikaranam-amazon commented Apr 13, 2022

Signed-off-by: Nicholas Walter Knize [email protected]
Signed-off-by: Sai Kumar [email protected]

Description

Adds a deprecated API to create a history operations snapshot from the translog.

Issues Resolved

#2482

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@saikaranam-amazon saikaranam-amazon marked this pull request as ready for review April 13, 2022 14:10
@saikaranam-amazon saikaranam-amazon requested a review from a team as a code owner April 13, 2022 14:10
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 8c783ef3f02ccd77cf2002b447a4bf969c2e982f
Log 4441

Reports 4441

@dblock
Copy link
Member

dblock commented Apr 13, 2022

I want to make sure we all agree with the complete proposal, especially that we will be doing #2872 for 2.1+ (moving CCR into core module so we can refactor the deprecated methods into CCR in 2.1), before merging this.

@dblock
Copy link
Member

dblock commented Apr 13, 2022

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 8c783ef3f02ccd77cf2002b447a4bf969c2e982f
Log 4451

Reports 4451

@saikaranam-amazon
Copy link
Member Author

Failed test case -

> Task :client:rest-high-level:asyncIntegTest

REPRODUCE WITH: ./gradlew ':client:rest-high-level:asyncIntegTest' --tests "org.opensearch.client.IndicesClientIT.testPutTemplateWithDeprecatedTemplateField" -Dtests.seed=1708D43D2AEEB3A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=de-AT -Dtests.timezone=SystemV/EST5 -Druntime.java=17

org.opensearch.client.IndicesClientIT > testPutTemplateWithDeprecatedTemplateField FAILED
    org.opensearch.client.WarningFailureException: method [PUT], host [http://[::1]:41543], URI [/_template/my-template?master_timeout=30s], status line [HTTP/1.1 200 OK]
    Warnings: [Parameter [master_timeout] is deprecated and will be removed in 3.0. To support inclusive language, please use [cluster_manager_timeout] instead., Deprecated field [template] used, replaced by [index_patterns]]
    {"acknowledged":true}
        at __randomizedtesting.SeedInfo.seed([1708D43D2AEEB3A:1B8DC88B8EC245FA]:0)
        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:346)
        at app//org.opensearch.client.RestClient$1.completed(RestClient.java:400)
        at app//org.opensearch.client.RestClient$1.completed(RestClient.java:396)
        at app//org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
        at app//org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
        at app//org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
        at app//org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
        at app//org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
        at app//org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
        at app//org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
        at app//org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
        at app//org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
        at app//org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
        at app//org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
        at app//org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
        at app//org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
        at app//org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
        at [email protected]/java.lang.Thread.run(Thread.java:833)

@tlfeng
Copy link
Collaborator

tlfeng commented Apr 14, 2022

Hi @saikaranam-amazon Please ignore this test failure, which has been fixed in 455fad3 . The deprecation message in the test was changed by mistake.
While that test failure will not fail the Gradle check build. You could also merge the current main branch into your feature branch to get the fix in your branch.

@saikaranam-amazon
Copy link
Member Author

@tlfeng Got it! thanks!

Adds a deprecated API to create a history operations snapshot from the translog.
This was superseded by always using the lucene index for peer recovery since the
index saves on costly storage but this API enables plugins to selectively use
the uncompressed xlog file for speed over disk savings. Note: this API will be
either completely removed or refactored in the next release.

Signed-off-by: Nicholas Walter Knize <[email protected]>
Signed-off-by: Sai Kumar <[email protected]>
@saikaranam-amazon
Copy link
Member Author

other failures

> Task :server:javadoc
/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/server/src/main/java/org/opensearch/index/engine/Engine.java:748: error: unknown tag: Deprecated
     * @Deprecated reading history operations from the translog file is deprecated and will be removed in the next release
       ^
/var/CITOOL/workflow/OpenSearch_CI/PR_Checks/Gradle_Check/search/server/src/main/java/org/opensearch/index/shard/IndexShard.java:2246: error: unknown tag: Deprecated
     * @Deprecated reading history operations from the translog file is deprecated and will be removed in the next release
       ^
/var/CITOOL/workflow/

Addressed in the latest revision

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success c3b7d04
Log 4471

Reports 4471

@saikaranam-amazon
Copy link
Member Author

saikaranam-amazon commented Apr 14, 2022

Please add the backport 2.0 label to merge to 2.0 branch.

@saikaranam-amazon
Copy link
Member Author

I want to make sure we all agree with the #2482 (comment), especially that we will be doing #2872 for 2.1+ (moving CCR into core module so we can refactor the deprecated methods into CCR in 2.1), before merging this.

Agreed and updated the issue - #2482

@dblock dblock requested a review from nknize April 14, 2022 18:45
Signed-off-by: Nicholas Walter Knize <[email protected]>
Copy link
Collaborator

@nknize nknize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approve w/ reservations.

@nknize nknize added deprecate v2.1.0 Issues and PRs related to version 2.1.0 backport 2.0 Backport to 2.0 branch labels Apr 14, 2022
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 263acf1
Log 4494

Reports 4494

@saikaranam-amazon
Copy link
Member Author

Thanks @nknize ! Could you please merge the PR?

  • I don't have access

@dblock dblock merged commit 2069aa3 into opensearch-project:2.x Apr 15, 2022
@dblock dblock added the backport 2.x Backport to 2.x branch label Apr 15, 2022
@dblock
Copy link
Member

dblock commented Apr 15, 2022

I merged and labeled to backport to 2.x as well

@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2886-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 2069aa372dddccadf8d5087fd76687f1a37cf2ec
# Push it to GitHub
git push --set-upstream origin backport/backport-2886-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2886-to-2.x.

nknize pushed a commit that referenced this pull request Apr 15, 2022
…2886)

* Add deprecated API for creating History Ops Snapshot from translog

Adds a deprecated API to create a history operations snapshot from the translog.
This was superseded by always using the lucene index for peer recovery since the
index saves on costly storage but this API enables plugins to selectively use
the uncompressed xlog file for speed over disk savings. Note: this API will be
either completely removed or refactored in the next release.

Signed-off-by: Nicholas Walter Knize <[email protected]>
Signed-off-by: Sai Kumar <[email protected]>

* fix javadoc casing for deprecated

Signed-off-by: Nicholas Walter Knize <[email protected]>

Co-authored-by: Nicholas Walter Knize <[email protected]>
(cherry picked from commit 2069aa3)
nknize pushed a commit that referenced this pull request Apr 15, 2022
…2886) (#2917)

* Add deprecated API for creating History Ops Snapshot from translog

Adds a deprecated API to create a history operations snapshot from the translog.
This was superseded by always using the lucene index for peer recovery since the
index saves on costly storage but this API enables plugins to selectively use
the uncompressed xlog file for speed over disk savings. Note: this API will be
either completely removed or refactored in the next release.

Signed-off-by: Nicholas Walter Knize <[email protected]>
Signed-off-by: Sai Kumar <[email protected]>

Co-authored-by: Nicholas Walter Knize <[email protected]>
(cherry picked from commit 2069aa3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport 2.0 Backport to 2.0 branch deprecate v2.1.0 Issues and PRs related to version 2.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants