Replace the deprecated IndexReader APIs with new storedFields() & termVectors()

[luyuncheng]

i try to use OpenSearch in production. and in our environment, we have tons of segments, tons of threads, tons of data.

i see there is deprecated API to retrieve document. These API would create many ThreadLocal StoreFields and make jvm can not GC. you can see the example in LUCENE#11987.

so i PR the code to replace the deprecated api with new storefields & termVectors API and reduced threadlocal creation.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/16419/
• CommitID: 8603945
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.search.backpressure.SearchBackpressureIT.testSearchShardTaskCancellationWithHighCpu

• URL: https://build.ci.opensearch.org/job/gradle-check/16420/
• CommitID: c18cd7f
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #7792 (11caaaf) into main (e1a4125) will decrease coverage by 0.03%.
The diff coverage is 57.14%.

@@             Coverage Diff              @@
##               main    #7792      +/-   ##
============================================
- Coverage     71.08%   71.05%   -0.03%     
+ Complexity    57237    57222      -15     
============================================
  Files          4758     4758              
  Lines        269844   269851       +7     
  Branches      39480    39481       +1     
============================================
- Hits         191819   191750      -69     
- Misses        61865    61969     +104     
+ Partials      16160    16132      -28     

Files Changed	Coverage Δ
...opensearch/common/lucene/search/XMoreLikeThis.java	36.61% <0.00%> (-0.30%)	⬇️
...earch/fetch/subphase/highlight/HighlightUtils.java	40.00% <0.00%> (ø)
...rg/opensearch/index/engine/TranslogLeafReader.java	38.00% <50.00%> (+0.50%)	⬆️
...ensearch/index/termvectors/TermVectorsService.java	17.58% <50.00%> (+0.36%)	⬆️
...ensearch/gateway/PersistedClusterStateService.java	88.41% <100.00%> (ø)
...va/org/opensearch/index/engine/InternalEngine.java	74.58% <100.00%> (-0.69%)	⬇️
...opensearch/index/engine/LuceneChangesSnapshot.java	85.41% <100.00%> (ø)
...java/org/opensearch/index/get/ShardGetService.java	60.89% <100.00%> (+0.64%)	⬆️
...rg/opensearch/index/shard/ShardSplittingQuery.java	74.04% <100.00%> (ø)
...n/java/org/opensearch/search/fetch/FetchPhase.java	38.95% <100.00%> (ø)
... and 2 more

... and 447 files with indirect coverage changes

[gashutos]

Thanks @luyuncheng just a minor comment but looks good to me !
There are no other places we use these 2 deprecated APIs right ?

Also tagging @dblock / @reta for additional reviews.

[luyuncheng]

I think the description of the change is a bit misleading, may be change it to:

We also need a CHANGELOG entry please (see please CHANGELOG.md), thank you

++ Updated
@reta thanks for the review

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/20967/
• CommitID: a29c62f
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/20968/
• CommitID: 6532cf7
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[reta]

Apologies @luyuncheng , conflicts :(, needs rebase with main

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/20982/
• CommitID: 949ebd4

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/20983/
• CommitID: 11caaaf

[luyuncheng]

conflicts :(, needs rebase with main

@reta Thanks. i sync upstream master into this branch, fixed conflicts. and run precommit locally seems success.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-7792-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 4fd6877eef7c9cb0b48268a7505506aeb33278a2
# Push it to GitHub
git push --set-upstream origin backport/backport-7792-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-7792-to-2.x.

[reta]

@luyuncheng sadly automation failed, could you please send manual backport to 2.x branch? thank you

[luyuncheng]

could you please send manual backport to 2.x branch? thank you

@reta OK, let me do it