Target platform

[simlei]

How come the target platform is subject to change

In the wake of Java 11 migration problems we had to add JAXB (xml binding) packages and a backend for it (glassfish)

Attempt no. 1: Eclipse orbit repositories:

f1d8715?diff=unified#diff-88c60f032be07d961c94842868be0d1aR20

However, this did not work, as Eclipse Orbit repositories host an incompatible version that uses a hardcoded, now-abandoned package namespace (something ...internal).

Therefore, we had to get the compatible, more up to date version (2.4+) from Maven. Shipping maven dependencies in OSGi is difficult to "cleanly" integrate into release engineering with Tycho and a normal Eclipse workspace development workflow. This is, because the production build goes through Tycho, the debug (day to day dev) build through Eclipse PDE. Both should be defined by a single definition to make the delivered products and the development version as identical as possible (else, mysterious bugs that get reported but cannot be reproduced). Writeups and analysis of this can be found here: here comparing five approaches and here. In a nutshell, the viable options are:

• Package individual jars as bundles in projects that are present in the workspace. We lose however any Maven <dependency>...</dependency> commodities, and have to carry around binaries in our source repository
• Go the full target platform/p2 way that resolves the Maven dependencies through a Maven plug-in and creates a P2 update site we can depend on in org.jcryptool.target.

The second approach is implemented as proof-of-concept in simlei/org.jcryptool.thirdparty and works. Resolving the JARs and packaging as target platform is done via different maven goals. Now, that #149 mandates that we ship our own JRE, and the best way seems to be as OSGi feature, this approach seems even more fitting. The JRE is of considerable size and I think we'd rather avoid putting it in our core repository (git history size would increase non-shallow repository clones with each update of the binaries).

Infrastructure

• If it works out well with simlei/org.jcryptool.thirdparty, it would be moved to a jcryptool organisation repository.
• In this project, there can actually be multiple p2 repositories. Compiling them into one unified p2 site is however unproblematic. For now, there is the org.jcryptool.thirdparty.java11migration.jaxb subproject that targets Java 11: Java XML bindings #151
• Right now, we host the p2 site with github pages. This is the best solution we have come up with for the duration. I think it is quite nice impromptu p2 hosting for various reasons:
  • reliable uptime at github
  • I have not seen transfer speed issues
  • The update site URL can be directly married to the github sources from which it is created (github pages url: <organization>.github.io/<repository>/<file path> versus github.com/<organization>/<repository>/<file path>).

Moving forward

• Is this kind of infrastructure acceptable for JCrypTool?

  • Third-party repository may be beneficial in stating our licences clearly, per repository. For example, "The JCrypTool product incorporates the 3rd party binaries defined in repository Y (specify lices there), unchanged, linked via standard Eclipse/Tycho build procedure. All contributions in our source repos X1 and X2 are subject to the contributor license X/License.md".
  • It does however add one more repository to be included in our CI build.

• make sure it all works with related issues in the forks (i.e. Java 11 migration #149 and Java 11: Java XML bindings #151)

• test that this all does not interfere with normal development [1]

• move p2-related projects to jcryptool organisation, change target definition references in the forks to the new urls, test again.

• merge subproject (plugin) changes from the forks that depend on target definition changes into jcryptool-organisation repositories

Standing issues

• [1] target definition updates are not automatically applied in a developer's workspace (as is sensible), which may lead to workspace builds not working after a git pull. This is normal for any RCP development. @grthor documented a workflow and the effects a target platform change had for a development clone on his PC: https://github.com/simlei/org.jcryptool.thirdparty/blob/master/doc/workspace-update-doc-1.pdf

A solution for the latter that could be considered in the future be a "JCT dev" plugin for Eclipse IDE that i.a. notifies the developer of target platform changes (checks in acceptable intervals or on common Team update actions like git pull).

[simlei]

This is how the provisory third-party repository manifests in the target definition right now: https://github.com/simlei/core/blob/892be3b19fdb8d9cfabfea2233d5f8ea6b0d888c/org.jcryptool.target/org.jcryptool.target.target#L22-L25

Corollary Manifest.MF changes:

grthor@021fc6c

[simlei]

The thirdparty update site project has grown as planned; the JRE features are now built and delivered through it as well. See https://github.com/org.jcryptool.thirdparty

There are now two update sites, at

• https://simlei.github.io/org.jcryptool.thirdparty/site -- this is the "index.html" hub, more may be added later
  (The following links do not have a html page, but Eclipse and our build process can resolve them.)
• https://simlei.github.io/org.jcryptool.thirdparty/site/m2bundle -- the <dependencies> stated on Maven Central artifacts (like javax xml bindings and transitive dependencies) are repackaged as bundles into this repository
• https://simlei.github.io/org.jcryptool.thirdparty/site/jre -- hosts the JRE features for the Linux/Windows/OSX which prevents our core repository to host these binaries.

The former update site for jaxb, https://simlei.github.io/org.jcryptool.thirdparty.java11migration.p2/site, stays online for a while even though it is now obsolete on my fork.

Credit for the move to the thirdparty repo working out well goes to @grthor with a feature-based (therefore Tycho-compatible) platform-aware solution to the JRE packaging problem.

The reasons to host this through github pages still stand -- quite convenient for the while being and reliable. Still, moving these p2 sites is mandatory in the future. At the same time, we should move this stuff from /simlei to /jcryptool.

Impression:

Build with Tycho was tested and succeeds, and packages the JREs the same way as happened in grthor/jre_bundle.
EDIT: The products do not launch successfully right now, though :( Also under linux I am now getting permission denied errors. I have found a promising workaround that would work for OSX as well :) https://www.eclipse.org/lists/tycho-user/msg08231.html

Please watch this space/branch in the next days if you want to work against the release 1.0 candidate that is most advanced: J11based-UpstreamCandidate

[simlei]

Everything works again, and I have good reason to believe it will work on OSX, too.
I actually have two alternatives to test for an OSX user! It sets the permissions now. Also, there is a "lib" linking happening in the first alternative, the second alternative is more minimal.

@be4 , @dschadow, @grthor please verify if you have the time! It is about starting up and having Java 11 running. More info is, where the download links are.

Here is a release on my fork of /core: https://github.com/simlei/core/releases/tag/v1.0-J11-1

Here are the relevant files in the JRE features for OSX:

• Candidate 1: three products: for each platform one, https://github.com/simlei/org.jcryptool.thirdparty/blob/5a24f29643f20e961a7d152adb0021a0c55efdb2/projects/org.jcryptool.thirdparty.jre/org.jcryptool.thirdparty.jre.macosx.feature/p2.inf

• One Alternative jre feature touchpoint for OSX -- candidate 2, https://github.com/simlei/org.jcryptool.thirdparty/blob/6cea2f9ce3cd86598086ba1e391ab1bc807306a8/projects/org.jcryptool.thirdparty.jre/org.jcryptool.thirdparty.jre.macosx.feature/p2.inf

More in #150 . In a nutshell: please extract and see if it starts. Then, verify with (Menu)Help>About>Installation Details>Configuration whether you have it running on Java 11 (the path should to java should be inside the extracted JCT "installation").