Modernize system test native makefile scripts

[Mesbah-Alam]

We have 3 native makefile scripts in system test:

1. Jck tests
2. Modularity test
3. Shared Classes tests

Each of these makefiles need to be updated every time we have a new platform to support.

We should modernize these scripts and see if they can figure out the platform (and the compiler to use) by themselves instead of PLATFORM being passed in.

FYI @llxia

[lumpfish]

The problem is working out whether to build 32 or 64 bit, wihich is dependent on the jdk being tested.

[Mesbah-Alam]

@lumpfish - I have been trying to run the compilation without passing the m32, m64 etc flags and the test seems to be compiling fine.

[lumpfish]

You mean you tested both a 32 an 64 bit jdk without using the flags and the tests ran OK on both?
Actually there is only 32 bit arm left now isn't there? Certainly if all jdks are now 64 bit the problem goes away.

[Mesbah-Alam]

You mean you tested both a 32 an 64 bit jdk without using the flags and the tests ran OK on both?

Tested successfully on all 64 bit platforms.

Among 32 bits, tested on win32 (internal Jenkins/Grinder/14131) and arm (https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/7622). The native system tests compile fine without the use of 32 bit specific flags on both.

Question is, what is the full set of 32 bit platforms that we support ?

A quick chat with @AdamBrousseau revealed that we support 32 bit on Java 7 and 8 only; and OpenJ9 Win32 JDK8. TKG based system tests do not support Java 7 (yet). So, that leaves us with Java 8 only.

A not-so-graceful way of course would be to deliver the change and fix any issue after.

@llxia : would you know if there is a way to test using Java 8 on 32 bit AIX, z/OS, Mac and Linux flavours? Currently the PLATFORM options on Grinder doesn't support these (it only has 32 bit Windows and ARM).

[smlambert]

While we currently focus on jdk8+, I know there was some discussion of attempting to cover jdk7 testing requirements as well. After our initial investigation of this (and seeing what additional work would be required), we decided we would not do this.

I think it is a fair approach to ensure we work well on the existing set of platforms listed in the PLATFORM_MAP which I believe was created with actual example builds in mind.

The platform map currently contains 6 32-bit platforms, arm_linux, x86-32_windows (which are openly built and supported), and ppc32_aix, ppc32_linux, s390_linux, x86-32_linux. For those last 4 cases, can you check espresso builds for IBM Java 8 builds for SDKs that you could use for testing? I would not want to remove support for those, if they still exist and given that there was an effort made to move to use AQA tests for those internal to IBM builds.

One other thing to note, there are 2 statements, "they compile fine" and "there are no newly introduced weird test failures because of compiler arg changes" to consider. So, for the 2 currently supported 32-bit platforms, as part of a PR check, a full set of tests should be run pre/post this change to check all things look sane. (and if the other 4 internal platform builds can be verified, even better, though not required from my perspective).

[Mesbah-Alam]

sanity.system started on:

• Linux ARM / Hotspot JDK8: https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/7698/ (green)

• Windows 32 bit / OpenJ9 JDK8: Internal/Grinder/14183 (green)

• PPC64LE Linux / OpenJ9 JDK11: Internal/14178 (green)

• X64 Windows / OpenJ9 JDK11: Internal/14182 (green)

• AARCH64 Linux / OpenJ9 JDK11: Internal/14177 (green)

• PPC64 AIX / OpenJ9 JDK11 : Internal/Grinder/14176 (green)

• X64 Linux / OpenJ9 JDK11: Internal/Grinder/14180 (green)

• X64 OSX / OpenJ9 JDK11: Internal/Grinder/14181 (green)

• s390x Linux / OpenJ9 JDK11: Internal/Grinder/14179 (green)

• PPC32 AIX : We are passing in -q${BitMode} to the compiler command on AIX (as without it compilation fails even on the 64 bit AIX). So, effectively the compiler options do not change for AIX and hence testing on 32 bit AIX is not needed.

• PPC32 Linux : Internal/Grinder_TKG/1173 (green)

• s390 Linux : Internal/Grinder_TKG/1162 (green)

• x86-32 Linux: Internal/Grinder_TKG/1164 (green)

• ZOS 390 64 : Internal/Grinder/14239 (green)

extended.system started on :

• Linux ARM / Hotspot JDK 8 : https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/7700/console (green)
• Windows 32bit / OpenJ9 JDK 8 : Grinder/14184 (green)

[Mesbah-Alam]

As the changes affect JCK too,sanity.jck started on internal Jenkins:

• Arch64 Linux: Grinder #14201 - 1 failure - jck-runtime-vm-constantpool_0
• PPC64LE Linux : Grinder #14202 - 1 failure - jck-runtime-vm-constantpool_0
• s390x Linux: Grinder #14203 - 1 failure - jck-runtime-vm-constantpool_0
• X64 Linux: Grinder #14209 - 1 failure - jck-runtime-vm-constantpool_0
• X64 Windows:Grinder #14257
• X86 Windows:Grinder #14212 (green)
• X64 OSX: Grinder #14206 - 1 failure - jck-runtime-vm-constantpool_0
• PPX64 AIX: Grinder #14207 - 1 failure - jck-runtime-vm-constantpool_0
• ZOS 390 64 : Grinder/14240/

[Mesbah-Alam]

The common jck-runtime-vm-constantpool_0 failure on all platforms, as reported in the comment above, doesn't seem to be related to the updates made for this issue though.

The failure occurs with test materials from master too (see: internal/Grinder/14210).

@JasonFengJ9 - could you please check if is this a known JCK failure? (A quick look up in OpenJ9 and Backlog repo doesn't reveal any obvious issue that may be related to this).

[JasonFengJ9]

The common jck-runtime-vm-constantpool_0 failure on all platforms

This is a known failure as per RTC 145068: JCK 11 failure : jck-runtime-vm-constantpool_0 - Segmentation error vmState=0x00040000 which was introduced around March 10th and fixed in next day.
Pls try a latest build.

[Mesbah-Alam]

This is a known failure as per RTC 145068

@JasonFengJ9 - Thanks for the information. Since it's a known failure, we can ignore it as far as verifying the PRs related to this issue is concerned. That said,

which was introduced around March 10th and fixed in next day.
Pls try a latest build.

All the above Grinders were run on March 13, 2021 and used OpenJ9 "nightlies" from Adopt. e.g.

12:57:43  OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.11+5-202103111808)
12:57:43  Eclipse OpenJ9 VM AdoptOpenJDK (build master-13936a4d3, JRE 11 Linux s390x-64-Bit Compressed References 20210311_911 (JIT enabled, AOT enabled)
12:57:43  OpenJ9   - 13936a4d3
12:57:43  OMR      - 2c0d1393c
12:57:43  JCL      - 8d7fbfd341 based on jdk-11.0.11+5)
12:57:44  =JAVA VERSION OUTPUT END=
12:57:44  =RELEASE INFO BEGIN=
12:57:44  IMPLEMENTOR="AdoptOpenJDK"
12:57:44  IMPLEMENTOR_VERSION="AdoptOpenJDK"
12:57:44  JAVA_VERSION="11.0.11"
12:57:44  JAVA_VERSION_DATE="2021-04-20"

It appears that the fix didn't make it into the above SDK and such (yet).

[JasonFengJ9]

OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.11+5-202103111808)

The JVM in question was built on March 11th which has OpenJ9 - 13936a4d3 and just missed eclipse-openj9/openj9@5b42292.
Next adopt build should fix the crash referred.

[smlambert]

3 PRs related to this issue are now merged:
#415
adoptium/openj9-systemtest#128
adoptium/STF#106

@Mesbah-Alam - thanks for the large amount of testing you did to get to this point. Believe this issue can now be closed.

[Mesbah-Alam]

Thanks for reviewing @smlambert and @llxia .

A good use case would now be to add a new platform (e.g. Alpine Linux) and see how seamlessly the system tests behave. Also, the code block we left intact for Solaris need testing too; something we can do once we have Solaris machines available.

[smlambert]

New-ish platforms to verify:

solaris_sparcv9
solaris_x86
Solaris machines are available, work to add testing is being done in this issue: adoptium/aqa-tests#799

alpine-linux:
The test machines (docker-based) for Alpine-linux were hitting an issue where a Perl module was not present. Work being done under issue adoptium/aqa-tests#2223

risc-v:
Work being done under adoptium/aqa-tests#2391

FYI @sxa