[distribution/archives] [Linux] [x64] Provide the variant of the distributions bundled with JRE

[reta]

Description

Explore the option to bundle JRE with OpenSearch, the archive (and image sizes) are ~50% smaller:

ll distribution/archives/jre-linux-tar/build/distributions/opensearch-min-3.0.0-SNAPSHOT-jre-linux-x64.tar.gz 
-rw-rw-r-- 1 opensearch opensearch 136M Jun 21 11:07 distribution/archives/jre-linux-tar/build/distributions/opensearch-min-3.0.0-SNAPSHOT-jre-linux-x64.tar.gz

ll distribution/archives/linux-tar/build/distributions/opensearch-min-3.0.0-SNAPSHOT-linux-x64.tar.gz 
-rw-rw-r-- 1 opensearch opensearch 235M Jun 21 11:06 distribution/archives/linux-tar/build/distributions/opensearch-min-3.0.0-SNAPSHOT-linux-x64.tar.gz

The server reports bundled JDK / JRE (but at the moment cannot distinguish between them).

[2023-07-26T13:11:16,942][INFO ][o.o.n.Node               ] [host] version[3.0.0-SNAPSHOT], pid[65472], build[tar/e350d0a93b7c0a6bf2be049f728547a966a6adc6/2023-07-26T16:41:53.199765Z], OS[Linux/6.2.0-26-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/20.0.2/20.0.2+9]
[2023-07-26T13:11:16,943][INFO ][o.o.n.Node               ] [host] JVM home [/distribution/archives/jre-linux-tar/build/install/opensearch-3.0.0-SNAPSHOT/jre], using bundled JDK/JRE [true]
[2023-07-26T13:11:16,944][INFO ][o.o.n.Node               ] [host] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.security.manager=allow, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-9970640782082341223, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.security.manager=allow, --add-modules=jdk.incubator.vector, -Djava.util.concurrent.ForkJoinPool.common.threadFactory=org.opensearch.secure_sm.SecuredForkJoinWorkerThreadFactory, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/distribution/archives/jre-linux-tar/build/install/opensearch-3.0.0-SNAPSHOT, -Dopensearch.path.conf=/distribution/archives/jre-linux-tar/build/install/opensearch-3.0.0-SNAPSHOT/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]

Related Issues

Closes #8899

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[reta]

@peterzhuamazon @dblock @andrross would be curious to hear your opinions, still in POC but packaging seems to be working as expected. Adding just Linux platform for now.

On the not so exciting part - I haven't found the way to backport it to 2.x cleanly since the build scripts (plugins) are having breaking changes.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

• URL: https://build.ci.opensearch.org/job/gradle-check/18091/
• CommitID: aa8ad88
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #8195 (d678e35) into main (b4b25dc) will decrease coverage by 0.01%.
The diff coverage is 31.51%.

@@             Coverage Diff              @@
##               main    #8195      +/-   ##
============================================
- Coverage     71.08%   71.07%   -0.01%     
- Complexity    57185    57256      +71     
============================================
  Files          4759     4763       +4     
  Lines        269885   269965      +80     
  Branches      39489    39498       +9     
============================================
+ Hits         191843   191877      +34     
- Misses        61835    61917      +82     
+ Partials      16207    16171      -36     

Files Changed	Coverage Δ
...ldSrc/src/main/java/org/opensearch/gradle/Jre.java	0.00% <0.00%> (ø)
.../java/org/opensearch/gradle/JreDownloadPlugin.java	0.00% <0.00%> (ø)
.../java/org/opensearch/gradle/VersionProperties.java	52.00% <0.00%> (-1.07%)	⬇️
...e/internal/InternalDistributionDownloadPlugin.java	0.00% <0.00%> (ø)
...a/org/opensearch/gradle/test/DistroTestPlugin.java	0.00% <0.00%> (ø)
.../main/java/org/opensearch/monitor/jvm/JvmInfo.java	75.54% <0.00%> (-0.28%)	⬇️
server/src/main/java/org/opensearch/node/Node.java	86.04% <0.00%> (ø)
...c/main/java/org/opensearch/gradle/JavaVariant.java	63.38% <63.38%> (ø)
.../org/opensearch/gradle/OpenSearchDistribution.java	73.91% <66.66%> (ø)
...in/java/org/opensearch/gradle/JavaPackageType.java	100.00% <100.00%> (ø)
... and 1 more

... and 459 files with indirect coverage changes

[peterzhuamazon]

Thanks @reta,

Copying my comments from another issue: opensearch-project/opensearch-build#3607 (comment)

Thanks @reta,

Seems like it is only tar for now.
We can have it in core until all the distributions in core updated with jre options, before we can fetch this into releases.
Would you mind giving possible timeline on this change @reta?

Wondering if we have time to bring it into 2.9, or 2.10.
(Edit: seems we can only bring this into 3.x per #8195 for now)
I do not think 1.x would have this change as 1.x is in maintenance mode now, and this is some what of a minor change.

Need more inputs: @dblock @bbarani

Thanks.

[reta]

Wondering if we have time to bring it into 2.9, or 2.10.
(Edit: seems we can only bring this into 3.x per #8195 for now)

Thanks @peterzhuamazon , yeah, this is correct, I think 2.9 could be feasible (we could find the way to bring it there, may not be pretty though), but looking what others \think, do we need that or may be not?

[peterzhuamazon]

Wondering if we have time to bring it into 2.9, or 2.10.
(Edit: seems we can only bring this into 3.x per #8195 for now)

Thanks @peterzhuamazon , yeah, this is correct, I think 2.9 could be feasible (we could find the way to bring it there, may not be pretty though), but looking what others \think, do we need that or may be not?

Hi @reta ,

This feels like require extensive testing as it is a big change imho.
So I do not want to treat it as just a size-reducing patch.
And it would be fine to introduce in 3.x.
In 2.x we are currently going through the Python Upgrade on build system, as well as OSD nodeJS upgrade to 18, which is a bit too much to include at the moment. Tho I am not closing the door of possibilities to bring to 2.x, just need proceed this with caution.

cc: @bbarani

Thanks.

[reta]

@peterzhuamazon @bbarani fyi, this is only change in core, not in shipping official distros yet, any concerns / objections from making it into 2.10.0?

[peterzhuamazon]

@peterzhuamazon @bbarani fyi, this is only change in core, not in shipping official distros yet, any concerns / objections from making it into 2.10.0?

Thanks @reta, also to confirm this is only tar/x64 right?

[reta]

@peterzhuamazon @bbarani fyi, this is only change in core, not in shipping official distros yet, any concerns / objections from making it into 2.10.0?

Thanks @reta, also to confirm this is only tar/x64 right?

For this change - yes, the meta issue will track each distribution separately

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/21115/
• CommitID: d6f8b20
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[peterzhuamazon]

Hi @reta ,

Would you help confirming there is proper test to this distribution / assemble task?

Thanks.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      2 org.opensearch.cluster.allocation.ClusterRerouteIT.testDelayWithALargeAmountOfShards

• URL: https://build.ci.opensearch.org/job/gradle-check/21116/
• CommitID: 70fc51c
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[reta]

Would you help confirming there is proper test to this distribution / assemble task?

@peterzhuamazon the tests for all distributions should be covered by #8924

[dblock]

This is very satisfying. Hit merge if you're satisfied @reta ? :)

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/21289/
• CommitID: d678e35

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8195-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 05b6647bf3af9556b68857ce14c131256bcd22f9
# Push it to GitHub
git push --set-upstream origin backport/backport-8195-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-8195-to-2.x.