Adding Java Image Builder Support #1675

theexplorist · 2019-07-22T22:15:44Z

PR Summary

This PR is in referance to GSoC project 'Adding More Options In Fabric8-Maven-Plugin for building images".

The PR refers to adding Java Image Builder Support to Fabric8-Maven-Plugin.
JIB Core module is used to add the functionality.

PR Checklist

codecov · 2019-07-23T00:13:25Z

Codecov Report

Merging #1675 into master will increase coverage by 0.43%.
The diff coverage is 59.89%.

@@             Coverage Diff              @@
##             master    #1675      +/-   ##
============================================
+ Coverage     33.49%   33.93%   +0.43%     
- Complexity     1097     1105       +8     
============================================
  Files           187      189       +2     
  Lines         10535    10704     +169     
  Branches       1724     1745      +21     
============================================
+ Hits           3529     3632     +103     
- Misses         6592     6650      +58     
- Partials        414      422       +8

devang-gaur

I'm sending fixes to these changes myself along with some other corrections.

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildService.java

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

devang-gaur · 2019-07-23T11:46:39Z

Hi @theexplorist ,

mvn -N license:format

You need to run the above command on your local clone to add licence header on the new code files being added.

Add a separate commit and push it.

devang-gaur · 2019-07-23T13:59:15Z

@lordofthejars @rohanKanojia

while reviewing, make sure you enable the isJib flag in pom.

Also, make sure you also try building using Jib with docker daemon as inactive.

theexplorist · 2019-07-23T16:21:07Z

@lordofthejars @rohanKanojia

while reviewing, make sure you enable the isJib flag in pom.

Also, make sure you also try building using Jib with docker daemon as inactive.

If you want to get just tarImage then just add isJib flag to be true in xmlconfig and the tarImage will be found at target/docker, its use case is if creds are not configured or wrong creds are configured.

If you want to see automatic registry push in action then you have to add a separate target Imagname in xmlconfig either under generator section to use the generator or under imageconfiguration. And the credentials must be configured as well.

devang-gaur · 2019-07-24T12:28:31Z

@theexplorist can you squash these commits into a single commit ?

rohanKanojia

This is my first round of review, I would look closely and try to run this and share more feedback whenever I get time. For now, please address these comments and add some documentation/tests.

core/pom.xml

rohanKanojia · 2019-07-25T16:12:08Z

core/src/main/java/io/fabric8/maven/core/access/ClusterAccess.java

@@ -116,7 +116,7 @@ public boolean isOpenShift(Logger log) {
        } catch (KubernetesClientException exp) {
            Throwable cause = exp.getCause();
            String prefix = cause instanceof UnknownHostException ? "Unknown host " : "";
-            log.warn("Cannot access cluster for detecting mode: %s%s",
+            log.warn("Cannot access OpenShift cluster for detecting mode: %s%s",


At this point, cluster can be either an OpenShift or kubernetes cluster. Why are you hardcoding Openshift?

Oops I added by mistake. @theexplorist can you take care of this ?

core/src/main/java/io/fabric8/maven/core/util/FatJarDetector.java

rohanKanojia · 2019-07-25T16:18:21Z

plugin/src/main/java/io/fabric8/maven/plugin/mojo/build/BuildMojo.java

    /**
-     * Enrichers used for enricher build objects
+     * Enrichers used for enri @Parameter(property = "fabric8.to", defaultValue = "")


This doesn't look right, maybe you commented some parameter?

No this is not needed , I will replace it!

rohanKanojia · 2019-07-25T16:19:54Z

samples/spring-boot-with-yaml/pom.xml

@@ -78,6 +79,17 @@
            </labels>
          </resources>

+          <generator>


why not add a separate sample for JIB demonstration?

Maybe we can make a seperate execution profile for Jib demonstration. A seperate sample isn't really required imo. @theexplorist make note.

rohanKanojia · 2019-07-25T16:20:42Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildService.java

+
+public class JibBuildService implements BuildService {
+
+    // TODO ADD LOGGING


Could you please resolve this TODO also? Maybe you forgot to resolve it?

Logging is implemented just comment is needed to be removed.

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

theexplorist · 2019-08-05T10:44:59Z

Thorntail sample is crashing even for normal docker build, the container starts and crashes!
Also the java options are barely added, because it always returns an empty list in thorntail generator which extends java exec.
https://github.com/fabric8io/fabric8-maven-plugin/blob/master/generator/java-exec/src/main/java/io/fabric8/maven/generator/javaexec/JavaExecGenerator.java#L172

https://github.com/fabric8io/fabric8-maven-plugin/blob/master/samples/thorntail/pom.xml#L96

devang-gaur · 2019-08-22T04:18:41Z

Thorntail sample is crashing even for normal docker build, the container starts and crashes!
Also the java options are barely added, because it always returns an empty list in thorntail generator which extends java exec.
https://github.com/fabric8io/fabric8-maven-plugin/blob/master/generator/java-exec/src/main/java/io/fabric8/maven/generator/javaexec/JavaExecGenerator.java#L172

https://github.com/fabric8io/fabric8-maven-plugin/blob/master/samples/thorntail/pom.xml#L96

The JAVA_OPTION system property is carried out in the generated yaml fragments and then injected into the container when run as part of a kubernetes/openshift deployment. The main cause of crashes is the AB_OFF system property... which needs to be set to true incase if thorntail sample right now

For reference, https://github.com/fabric8io-images/java/blob/master/images/alpine/openjdk8/jdk/README.md

rohanKanojia

Apart from the comments, I faced problems while trying to build the samples provided in this PR. It only works when image name has your docker hub username attached to image section. Plus I'm still confused about why we are pushing image during the build goal.

rohanKanojia · 2019-08-27T08:41:01Z

samples/spring-boot-with-yaml/pom.xml

+                <config>
+                  <spring-boot>
+                    <!-- Target Image name to which Jib Container will build to. Change ${image.user} with registry username -->
+                    <name>${image.user}/${project.artifactId}:${project.version}</name>


where is image.user defined?

image.user needs to be changed with the registry username in case you want to build a registry image

rohanKanojia · 2019-08-27T08:48:19Z

samples/spring-boot-without-route/pom.xml

+                    <plugin>
+                        <groupId>io.fabric8</groupId>
+                        <artifactId>fabric8-maven-plugin</artifactId>
+                        <version>4.2-SNAPSHOT</version>


update version to 4.3-SNAPSHOT, or use ${project.version}

rohanKanojia · 2019-08-27T08:51:41Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildServiceUtil.java

+import io.fabric8.maven.core.util.FatJarDetector;
+import io.fabric8.maven.docker.access.AuthConfig;
+import io.fabric8.maven.docker.config.BuildImageConfiguration;
+import io.fabric8.maven.docker.config.ImageConfiguration;


unused import

rohanKanojia · 2019-08-27T08:54:42Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildServiceUtil.java

+            contBuild = contBuild.setEntrypoint(entrypointList);
+        }
+
+        if (credential != null) {


what happens when credential is null? I see username and password are crucial dependencies for image generation.

A tar image is generated locally

rohanKanojia · 2019-08-27T08:59:12Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildServiceUtil.java

+        try {
+            Containerizer containerizer = Containerizer.to(registryImage.addCredential(username, password));
+            JibContainer jibRegistryImageContainer = contBuild.containerize(containerizer);
+            log.info("Image %s successfully built and pushed.", targetImage);


In my opinion, the default behavior should be generation of image locally. Why are we doing that as a fallback? Shouldn't image push step go in PushMojo ?

Jib auto pushes the images to registry if the creds are verified! That's how it works! If registry is not verified a tar archive is generated and then you can use that archive for push goal!

That's what a RegistryImage is meant to do in JIB, that's why they have this type of image!

I don't like the idea of pushing image during build goal as it does not align with mojo's standard functionality. I would prefer finding some offline solution if possible.

We have the offline solution, Jib supports three kinds
1.DockerDaemonImage - We don't need it as we are making it daemonless.
2.RegistryImage - It will do auto push that's Jib's functionality. That's why it is RegistryImage.
3.TarImage - This option is already added. Here you can use it for push goal.No need to configure creds etc here. You will get a tar archive.

Does the user get to choose these build options? If not, we can provide some way to override the default behavior. The default behavior should be building tarball.

rohanKanojia · 2019-08-27T09:20:10Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

+import java.util.List;
+import java.util.Map;
+
+public class JibBuildConfiguration {


I'd love to see a JavaDoc for each of these classes explaining how they fit together for the benefit of future developers.

rohanKanojia · 2019-08-27T09:22:25Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

+
+    public static class Builder {
+        private final JibBuildConfiguration configutil;
+        private final Logger logger;


What's the purpose of having a logger field inside a builder?

Its need to consumed by a static method of FatJarDetector https://github.com/fabric8io/fabric8-maven-plugin/pull/1675/files/aa9ed76e932483ddeaaaa471698f99b20410237c#diff-15957aeb41cce2019190e8b80fb7180eR119

rohanKanojia · 2019-08-27T09:23:27Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

+
+    public Arguments getEntryPoint() {return entrypoint;}
+
+    public String getTargetDir() {return targetDir;}


nit, spaces between braces and expressions. i.e { return targetDir; }

rohanKanojia · 2019-08-27T09:26:11Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildConfiguration.java

+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+package io.fabric8.maven.core.service.kubernetes;


Shouldn't this class go to io.fabric8.maven.core.config package?

rohanKanojia · 2019-08-27T09:26:56Z

core/src/main/java/io/fabric8/maven/core/service/kubernetes/JibBuildServiceUtil.java

+ * implied.  See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+package io.fabric8.maven.core.service.kubernetes;


Shouldn't this class go to io.fabric8.maven.core.util package?

Its not a generic utility. So I thought its okay for it to be here

devang-gaur · 2019-08-27T09:45:16Z

When the system is offline, Jib build crashes while building image tarball. I think we're blocked on this GoogleContainerTools/jib#1881 right now.
Everything else LGTM for now.

devang-gaur · 2019-09-03T20:20:33Z

When the system is offline, Jib build crashes while building image tarball. I think we're blocked on this GoogleContainerTools/jib#1881 right now.
Everything else LGTM for now.

Update, Jib provides an offline mode as well. So our use case is sorted for now.

rohanKanojia · 2019-09-10T13:40:40Z

core/src/main/java/io/fabric8/maven/core/util/JibBuildServiceUtil.java

+            log.warn("Registry Exception occured : %s", re.getMessage());
+            log.warn("Credentials are probably either not configured or are incorrect.");
+            log.info("Building Image Tarball at %s.", imageTarName);
+            JibContainer jibContainer =  buildContainer(contBuild, tarImage, log, false);


@dev-gaur : Can we make this a flag so that the user can also override in case he wants an offline build?

rohanKanojia

Tested and seems to be working fine for me.

Nice work @theexplorist @dev-gaur !

devang-gaur · 2019-09-17T06:43:06Z

#1536

This ports Jib support added in FMP in fabric8io/fabric8-maven-plugin#1675 with better logging.

This pull request shall port the following PRs: fabric8io/fabric8-maven-plugin#1675 fabric8io/fabric8-maven-plugin#1734 fabric8io/fabric8-maven-plugin#1739 fabric8io/fabric8-maven-plugin#1740

devang-gaur added Epic A larger user story pr/wip Work in Progress, do not merge labels Jul 23, 2019

devang-gaur suggested changes Jul 23, 2019

View reviewed changes

devang-gaur requested review from rohanKanojia and lordofthejars July 23, 2019 11:48

devang-gaur force-pushed the pr-1662 branch from 16c682a to 16c2af4 Compare July 23, 2019 13:48

devang-gaur self-assigned this Jul 23, 2019

rohanKanojia suggested changes Jul 25, 2019

View reviewed changes

theexplorist force-pushed the pr-1662 branch 3 times, most recently from 1cbc40c to 6cf8bc9 Compare July 26, 2019 10:28

theexplorist force-pushed the pr-1662 branch 2 times, most recently from 534ce29 to 9b59441 Compare August 2, 2019 14:12

theexplorist mentioned this pull request Aug 8, 2019

JAVA_OPTIONS do not get added in the thorntail generator #1688

Closed

rohanKanojia added the gsoc candidate for GSoC label Aug 26, 2019

rohanKanojia suggested changes Aug 27, 2019

View reviewed changes

theexplorist force-pushed the pr-1662 branch from 56df1ba to 138a289 Compare September 3, 2019 17:17

devang-gaur force-pushed the pr-1662 branch from 138a289 to a2f5b8c Compare September 3, 2019 20:18

devang-gaur requested a review from rohanKanojia September 3, 2019 20:20

devang-gaur force-pushed the pr-1662 branch from a2f5b8c to 73ea5c6 Compare September 4, 2019 13:29

devang-gaur removed the pr/wip Work in Progress, do not merge label Sep 9, 2019

rohanKanojia force-pushed the pr-1662 branch from 73ea5c6 to c7b87a7 Compare September 10, 2019 13:35

rohanKanojia reviewed Sep 10, 2019

View reviewed changes

rohanKanojia approved these changes Sep 10, 2019

View reviewed changes

Add Java Image Builder Support

e73884d

devang-gaur force-pushed the pr-1662 branch from eac7056 to e73884d Compare September 11, 2019 10:42

devang-gaur merged commit 7795450 into fabric8io:master Sep 11, 2019

rohanKanojia added a commit to rohanKanojia/docker-maven-plugin that referenced this pull request Oct 12, 2019

Support for JIB(Java Image Builder) mode in dmp

0ce5eac

This ports Jib support added in FMP in fabric8io/fabric8-maven-plugin#1675 with better logging.

rohanKanojia mentioned this pull request Oct 12, 2019

Support for JIB(Java Image Builder) mode in dmp fabric8io/docker-maven-plugin#1277

Merged

rohanKanojia added a commit to rohanKanojia/docker-maven-plugin that referenced this pull request Oct 12, 2019

Support for JIB(Java Image Builder) mode in dmp

c4af0d5

This ports Jib support added in FMP in fabric8io/fabric8-maven-plugin#1675 with better logging.

rohanKanojia added a commit to rohanKanojia/docker-maven-plugin that referenced this pull request Oct 14, 2019

Support for JIB(Java Image Builder) mode in dmp

828952e

This ports Jib support added in FMP in fabric8io/fabric8-maven-plugin#1675 with better logging.

rohanKanojia mentioned this pull request Oct 27, 2019

JibBuildConfiguration ignores BuildImageConfiguration.registry #1738

Closed

rohanKanojia mentioned this pull request Nov 8, 2019

Ported JIB implementation eclipse-jkube/jkube#14

Closed


		public class JibBuildService implements BuildService {

		// TODO ADD LOGGING


		public Arguments getEntryPoint() {return entrypoint;}

		public String getTargetDir() {return targetDir;}

Adding Java Image Builder Support #1675

Adding Java Image Builder Support #1675

Conversation

theexplorist commented Jul 22, 2019 • edited by rohanKanojia Loading

PR Summary

PR Checklist

codecov bot commented Jul 23, 2019 • edited Loading

Codecov Report

devang-gaur left a comment

Choose a reason for hiding this comment

devang-gaur commented Jul 23, 2019

devang-gaur commented Jul 23, 2019

theexplorist commented Jul 23, 2019

devang-gaur commented Jul 24, 2019

rohanKanojia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theexplorist commented Aug 5, 2019 • edited Loading

devang-gaur commented Aug 22, 2019

rohanKanojia left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theexplorist Aug 29, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theexplorist Aug 29, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devang-gaur commented Aug 27, 2019

devang-gaur commented Sep 3, 2019

Choose a reason for hiding this comment

rohanKanojia left a comment

Choose a reason for hiding this comment

devang-gaur commented Sep 17, 2019

theexplorist commented Jul 22, 2019 •

edited by rohanKanojia

Loading

codecov bot commented Jul 23, 2019 •

edited

Loading

theexplorist commented Aug 5, 2019 •

edited

Loading

theexplorist Aug 29, 2019 •

edited

Loading

theexplorist Aug 29, 2019 •

edited

Loading