[improve][docker] Switch to Temurin JDK #17129

nodece · 2022-08-17T03:45:39Z

Signed-off-by: Zixuan Liu [email protected]

Motivation

I noticed we are using OpenJDK in our Docker image, I suggest that we switch to the Temurin JDK, because our CI runs on the Temurin JDK, we need to keep the same JDK everywhere to avoid unexpected problems.

The Temurin JDK is OpenJDK distribution from Adoptium, the old JDK from Ubuntu, they should all be built on the OpenJDK open source project, and should be fully compatible.

The Temurin JDK is safe and reliable.

The Eclipse Temurin™ project provides code and processes that support the building of runtime binaries and associated technologies that are high performance, enterprise-caliber, cross-platform, open-source licensed, and Java SE TCK-tested for general use across the Java ecosystem.

https://projects.eclipse.org/projects/adoptium.temurin

If we need to discuss this more, see https://lists.apache.org/thread/t8l4r36vdyo7stkwcml7p1s22f466x8o.

Modifications

Update test image
Update build image
Update pulsar image

Documentation

Check the box below or label this PR directly.

Need to update docs?

doc-required
(Your PR needs to update docs and you will update later)
doc-not-needed
(Please explain why)
doc
(Your PR contains doc changes)
doc-complete
(Docs have been already added)

Signed-off-by: Zixuan Liu <[email protected]>

tisonkun

Thank you!

docker/pulsar/Dockerfile

tisonkun

To continue #12944, there're some code conflicts with this patch. cc @merlimat @codelipenghui @lhotari @eolivelli could you help with reviewing this patch to unblock further docker image building improvements?

mattisonchao · 2022-08-23T07:31:42Z

Question:
Do we have some compatible issues with two different JDKs?
I wonder if it will affect some users who use old JDK and upgrade with new JDK.

And I think this is a big change and may affect the production docker image. We have to notice PMCs approve it.
Thanks a lot!

nodece · 2022-08-23T08:31:58Z

Do we have some compatible issues with two different JDKs?

All are based on OpenJDK opensource code, and all should be fully compatible.

I wonder if it will affect some users who use old JDK and upgrade with new JDK.

Our tests have been using Temurin JDK for some time, so I don't think there is any impact, Temurin JDK will only be more stable.

And I think this is a big change and may affect the production docker image. We have to notice PMCs approve it.

I agreed with you!

tisonkun · 2022-08-30T13:02:48Z

FYI - we may try to use eclipse-temurin:17-focal image :)

nodece · 2022-08-31T02:18:41Z

FYI - we may try to use eclipse-temurin:17-focal image :)

Using this image looks like a good idea, but there will be a lot of big changes.

tisonkun · 2022-08-31T02:21:41Z

@nodece Yep. Not a requirement for this patch, just for your information :)

nodece · 2022-08-31T07:09:13Z

Hi @codelipenghui, @Technoboy- , @michaeljmarshall, @nicoloboschi, @merlimat, @lhotari, and @eolivelli, could you help review this PR? This is a big change for the Pulsar.

lhotari

Good work @nodece

michaeljmarshall · 2022-09-10T16:27:33Z

docker/pulsar/Dockerfile

+     && apt-get -y install temurin-17-jdk
+
+# Cleanup apt
+RUN apt-get -y --purge autoremove \


Don't we need to run this in the same command that we run our apt-get installations in order to get the benefit of a smaller docker image? Otherwise, we'll add files in one layer and remove them in another.

Good point. Would you like to prepare a patch and show the difference in size?

Included in #17733. You can give it a review :)

Revert as if I perform this command merge, the build will fail with returned a non-zero code: 100.

Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 4378856) Signed-off-by: Zixuan Liu <[email protected]>

* [Dockerfile] Enable retries for apt-get when building Pulsar docker image (apache#14513) - also reduce default timeout to 30 seconds - prevents issues where apt repository doesn't respond (cherry picked from commit d3f6fe3) * PIP-155: Removed Python 2 support (apache#15376) * Remove Pulsar Client Build for Python 2.7 * Remove outdated homebrew files (source of truth is upstream homebrew) * Remove Python 2.7 build references; print error in some cases * Update python client tests to run with python client for python 3.5m * PIP-155: Removed Python 2 support * Fixed invocation in pulsar-build image * Fixed clang-format-10 indent differences * Fixed script invocation with wrong python * We don't need to rebuild the manylinux image each time * Fixed image name * Reverted back to use newer protobuf * Fixed image name * Fixed missing python3 in centos:7 image * Use python3 for gtest-parallel * Show bash commands in docker-tests.sh * Fixed gh action issue with git directory permissions * Fixed python to 3 * Fixed custom_logger_test.py * Fixed path in run_python_instance_tests.sh * Function runtime should use python3 * Fixed function runtime test python expectation * Fixed presto worker launcher * Fixed notes on how to format C++ code Co-authored-by: Michael Marshall <[email protected]> (cherry picked from commit 2b2e0c5) Signed-off-by: Zixuan Liu <[email protected]> * [improve][docker] Switch to Temurin JDK (apache#17129) Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 4378856) Signed-off-by: Zixuan Liu <[email protected]> * [refactor][ci] Build the docker image with docker-maven-plugin (apache#17148) (cherry picked from commit a68b58d) Signed-off-by: Zixuan Liu <[email protected]> * [feat][build] Support ARM64-based docker images (apache#17733) (cherry picked from commit 9a2aeb2) Signed-off-by: Zixuan Liu <[email protected]> * PIP-209: Removed C++/Python clients from main repo (apache#17881) * PIP-209: Removed C++/Python clients from main repo * Removed python directory from Docekrfile * Fixed python client version argument scoping * Fixed handling of pulsar.functions.serde (cherry picked from commit f3c547b) Signed-off-by: Zixuan Liu <[email protected]> * [improve][build] Avoid building image multiple times (apache#17208) Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 79a97a9) * [improve] Allow to build and push multi-arch Docker images (apache#19432) Co-authored-by: Lari Hotari <[email protected]> Co-authored-by: Yong Zhang <[email protected]> Co-authored-by: Zixuan Liu <[email protected]> Co-authored-by: tison <[email protected]> (cherry picked from commit 4190e40) Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Fix publish image script (apache#20305) Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 94c7bf3) Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Fix the pulsar-all image may use the wrong upstream image (apache#20435) Signed-off-by: Zike Yang <[email protected]> Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit d7f3558) Signed-off-by: Zixuan Liu <[email protected]> * [feat][build] Adapt to Python client to be compatible with ARM arch Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Configure git-commit-id-plugin to skip git describe (apache#20550) (cherry picked from commit 05f7e62) * [improve][misc] Include native epoll library for Netty for arm64 From apache#22319 Signed-off-by: Zixuan Liu <[email protected]> * [fix][misc] Rename all shaded Netty native libraries From apache#22415 Signed-off-by: Zixuan Liu <[email protected]> * [cleanup][build] Cleanup -Ddocker.nocache=true Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Fix ubuntu mirror Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Fix license Signed-off-by: Zixuan Liu <[email protected]> * [fix][build] Downgrade docker-maven to 0.43.3 Signed-off-by: Zixuan Liu <[email protected]> --------- Signed-off-by: Zixuan Liu <[email protected]> Signed-off-by: Zike Yang <[email protected]> Co-authored-by: Lari Hotari <[email protected]> Co-authored-by: Matteo Merli <[email protected]> Co-authored-by: Michael Marshall <[email protected]> Co-authored-by: tison <[email protected]> Co-authored-by: Yong Zhang <[email protected]> Co-authored-by: Zike Yang <[email protected]> Co-authored-by: Lari Hotari <[email protected]>

[improve][docker] Switch to Temurin JDK

d7cd70e

Signed-off-by: Zixuan Liu <[email protected]>

tisonkun approved these changes Aug 17, 2022

View reviewed changes

github-actions bot added the doc-not-needed Your PR changes do not impact docs label Aug 17, 2022

RobertIndie approved these changes Aug 17, 2022

View reviewed changes

nodece requested review from codelipenghui, michaeljmarshall and nicoloboschi August 17, 2022 14:07

tisonkun reviewed Aug 18, 2022

View reviewed changes

docker/pulsar/Dockerfile Show resolved Hide resolved

nodece requested review from merlimat, lhotari and eolivelli August 18, 2022 03:19

Merge branch 'master' into switch-temurin-jdk

613147f

tisonkun reviewed Aug 23, 2022

View reviewed changes

nodece requested a review from Technoboy- August 23, 2022 06:40

mattisonchao requested a review from 315157973 August 23, 2022 07:36

nodece added the area/build label Aug 31, 2022

lhotari approved these changes Aug 31, 2022

View reviewed changes

nodece merged commit 4378856 into apache:master Aug 31, 2022

nodece added the release/2.11.1 label Aug 31, 2022

michaeljmarshall reviewed Sep 10, 2022

View reviewed changes

tisonkun mentioned this pull request Sep 12, 2022

[fix][CI] Don't run "Pulsar CI checks completed" too early and fix disk space issue in saving docker image #17584

Merged

4 tasks

Technoboy- assigned nodece Jan 22, 2024

nodece added a commit to nodece/pulsar that referenced this pull request Apr 28, 2024

[improve][docker] Switch to Temurin JDK (apache#17129)

961e8e5

Signed-off-by: Zixuan Liu <[email protected]> (cherry picked from commit 4378856) Signed-off-by: Zixuan Liu <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[improve][docker] Switch to Temurin JDK #17129

[improve][docker] Switch to Temurin JDK #17129

nodece commented Aug 17, 2022

tisonkun left a comment

tisonkun left a comment

mattisonchao commented Aug 23, 2022 •

edited

Loading

nodece commented Aug 23, 2022

tisonkun commented Aug 30, 2022 •

edited

Loading

nodece commented Aug 31, 2022

tisonkun commented Aug 31, 2022

nodece commented Aug 31, 2022

lhotari left a comment

michaeljmarshall Sep 10, 2022

tisonkun Sep 10, 2022

tisonkun Sep 20, 2022

tisonkun Sep 20, 2022

[improve][docker] Switch to Temurin JDK #17129

[improve][docker] Switch to Temurin JDK #17129

Conversation

nodece commented Aug 17, 2022

Motivation

Modifications

Documentation

tisonkun left a comment

Choose a reason for hiding this comment

tisonkun left a comment

Choose a reason for hiding this comment

mattisonchao commented Aug 23, 2022 • edited Loading

nodece commented Aug 23, 2022

tisonkun commented Aug 30, 2022 • edited Loading

nodece commented Aug 31, 2022

tisonkun commented Aug 31, 2022

nodece commented Aug 31, 2022

lhotari left a comment

Choose a reason for hiding this comment

michaeljmarshall Sep 10, 2022

Choose a reason for hiding this comment

tisonkun Sep 10, 2022

Choose a reason for hiding this comment

tisonkun Sep 20, 2022

Choose a reason for hiding this comment

tisonkun Sep 20, 2022

Choose a reason for hiding this comment

mattisonchao commented Aug 23, 2022 •

edited

Loading

tisonkun commented Aug 30, 2022 •

edited

Loading