Added jtreg-buffer testsuite #5308

judovana · 2024-05-10T11:53:04Z

judovana · 2024-05-10T12:22:39Z

s390x https://ci.adoptium.net/view/Test_grinder/job/Grinder/9927
aarch64 https://ci.adoptium.net/view/Test_grinder/job/Grinder/9928/

functional/jtreg-buffer/README.md

karianna

Need to understand why :-)

Co-authored-by: Martijn Verburg <[email protected]>

judovana · 2024-05-14T06:03:56Z

Need to understand why :-)

Yah. That it honest question, and not easy one indeed! Various reasons, from dummy ones like: Original author quit, original author forget, original author was lazy or simply not brave enough to push it. And later, no one had time to take it over. Not exactly an good excuse, but is covering about half of the tests. I would silently nod in agreeing to anybody refusing to accept this test set for this reason. But I tried to do quite a few in past, and burned out on it. Other reasons are weird corner cases, which do not run everywhere. Most of them are currenly labeled as var.rh.jdk although better would be rhel.and.fedora. Others may get this label once I see them to fail on some Adoptium combination I had not yet tried. Some have security manager. And as that is going to be dropped, but not dropped, or at least not soon, usptream is not accepting new tests with SM. Soem of them were upstreamed in form, that the intersection of covered code was not identical, or illegible for backport. So we decided to keep also orignal version (see speed)
Main of all is speed. upstreaming test is sometimes lengthy (and triple it when you porting it down to the JDK where you need it most), and if one need a testcase asap... That is actually why this repo was ever created. Howerv... from there, is it super easy to slip... goto "dummy ones"; And thats why it is not empty, but actually grown a bit.

judovana · 2024-05-22T14:32:00Z

I had recalled, that I was tracking the upstreaming. Its part of the repo, you can see it via githubhack: https://rawcdn.githack.com/rh-openjdk/jtreg-buffer/581161a13dd731f9eac1f89e5164884a77465bca/dupes.html

the removed are the tests which were upstreamed

sophia-guo · 2024-05-27T20:25:12Z

Is this a temporary solution ( adding to functional instead of openjdk) ? What's your suggestion @smlambert ?

smlambert · 2024-05-27T20:38:32Z

I would prefer tests that were intended to be upstreamed to openjdk to be put into the openjdk 'group' in AQAvit rather than functional to avoid having to move them later.

judovana · 2024-05-28T14:17:17Z

to ensure I understand correctly:)

You want me to move
https://github.com/adoptium/aqa-tests/pull/5308/files#diff-0a96f2e0342ab46eec92d5dfe1dc99c8d51f4a0de6fb362a5e9718bd53ad45f2R17-R59 to https://github.com/adoptium/aqa-tests/blob/master/openjdk/playlist.xml with s; <group>functional</group>;<group>openjdk</group>;g
and
https://github.com/adoptium/aqa-tests/pull/5308/files#diff-a9c20df99ab7ab76b335ff672d9d7a9b3af38dedf6f6a5642b74feafd25197b2R25-R70 to https://github.com/adoptium/aqa-tests/blob/master/openjdk/build.xml with some modifications to make ti work. oook?

smlambert · 2024-06-05T14:50:49Z

functional/jtreg-buffer/playlist.xml

+               <group>functional</group>
+          </groups>
+     </test>
+     <test>


Catch me up on the logic of this, I assumed that we have the 2 targets jtreg-buffer and jtreg-buffer_jtreg to allow vendors to run all tests on all platforms without the run.sh. The main issue with the run.sh was not because readlink -f misbehaves, but that run.sh may at any point start including other behaviour that is unwanted.

The current <disable> block for each target, means that these tests will run twice for platforms not defined by <platform>.

In this case it would be https://github.com/rh-openjdk/jtreg-buffer/blob/main/run.sh#L121 which would otherwise be replciated everywhere. run.sh is from rh-openjdk organization, so it should really not do anything bad.

(and the https://github.com/rh-openjdk/jtreg-buffer/blob/main/run.sh#L111, which is disabled in aqavit, but hsoudl be allowed to be enabled)

smlambert · 2024-06-05T17:07:41Z

JDK8 sparcv9_solaris: https://ci.adoptium.net/view/Test_grinder/job/Grinder/10231 - 9 failures

FAILED: reproducers/1759335/keytool-broken-fips.sh
FAILED: reproducers/1780335/fips-load-p11-kit-trust.sh
FAILED: reproducers/2036462/Test.java
FAILED: reproducers/6656625/Test.java
FAILED: reproducers/6657133/Test.java
FAILED: reproducers/6804996/Test.java
FAILED: reproducers/6804998/Test.java
FAILED: reproducers/6823373/Test.java
FAILED: reproducers/7122141/Main.java

JDK11 aarch64_linux: https://ci.adoptium.net/view/Test_grinder/job/Grinder/10221
JDK11 aarch64_mac: https://ci.adoptium.net/view/Test_grinder/job/Grinder/10222/ - 2 failures exit code 134, perhaps more info in jtr file?

11:21:26  FAILED: test/reproducers/6804996/Test.java
11:21:27  FAILED: test/reproducers/6823373/Test.java

JDK11 x86-64_linux: https://ci.adoptium.net/view/Test_grinder/job/Grinder/10223
JDK21 x86-64_windows: https://ci.adoptium.net/view/Test_grinder/job/Grinder/10230

judovana · 2024-06-05T17:12:03Z

That is the consequence of missing -javaoption:-Djava.security.manager=allow" as that is the direct run without run.sh

Missing info in jtr.xmls is due to missing openjdk/jtreg@8c105f2 in used jtreg.jar

judovana · 2024-06-05T17:18:42Z

That is the consequence of missing -javaoption:-Djava.security.manager=allow" as that is the direct run without run.sh

openjdk version "11.0.24-beta" 2024-07-16
OpenJDK Runtime Environment Temurin-11.0.24+5-202405311316 (build 11.0.24-beta+5-ea)
OpenJDK 64-Bit Server VM Temurin-11.0.24+5-202405311316 (build 11.0.24-beta+5-ea, mixed mode)
*** Terminating app due to uncaught exception 'NSRangeException', reason: '*** -[__NSArray0 objectAtIndex:]: index 0 beyond bounds for empty array'
*** First throw call stack:
(
	0   CoreFoundation                      0x0000000182aae800 __exceptionPreprocess + 176
	1   libobjc.A.dylib                     0x00000001825a5eb4 objc_exception_throw + 60
	2   CoreFoundation                      0x0000000182ad7c70 CFArrayApply + 0
	3   libsplashscreen.dylib               0x0000000100478a24 SplashNSScreen + 36
	4   libsplashscreen.dylib               0x0000000100477970 __SplashGetScaledImageName_block_invoke + 44
	5   Foundation                          0x0000000183b5e2d8 __NSThreadPerformPerform + 264
	6   CoreFoundation                      0x0000000182a39cfc __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 28
	7   CoreFoundation                      0x0000000182a39c90 __CFRunLoopDoSource0 + 176
	8   CoreFoundation                      0x0000000182a39a00 __CFRunLoopDoSources0 + 244
	9   CoreFoundation                      0x0000000182a385f0 __CFRunLoopRun + 828
	10  CoreFoundation                      0x0000000182a37c5c CFRunLoopRunSpecific + 608
	11  libjli.dylib                        0x0000000100559af8 CreateExecutionEnvironment + 400
	12  libjli.dylib                        0x0000000100555f54 JLI_Launch + 1196
	13  java                                0x00000001000d3b90 main + 396
	14  dyld                                0x00000001825e10e0 start + 2360
)
libc++abi: terminating due to uncaught exception of type NSException
/Users/admin/workspace/workspace/Grinder/aqa-tests/functional/jtreg-buffer/jtreg-buffer/test/reproducers/6823373/Test.sh: line 20:  1333 Abort trap: 6           $JAVA -splash:splash.jpg -showversion Test > LOG

Sure it is not that allow. Allow is jdk 17 and up. My apologise. It is late here.

judovana · 2024-06-10T12:13:41Z

It seems that all except mac and spark solaris is passing. I will disable those two tests on aaarch64 mac I think. and that 9 on jdk8 sparcv9 solaris

judovana · 2024-06-10T12:16:04Z

Looking to the set of failures, this suite would be nice to be forzen on hash/tag

judovana · 2024-06-28T08:48:32Z

It seems that this one is not good enough for geenral testing. It seems there will be introduced vendor-specific pipeline, so twill go there. Keeping open for now..

judovana · 2024-07-23T15:35:52Z

This will be moved to the vendor-specific pipeline, thus commiting elsewhere: rh-openjdk/jtreg-buffer#23

judovana force-pushed the jtregBufferII branch from 6487a36 to 5e13b5e Compare May 10, 2024 11:58

Added jtreg-buffer testsuite

58208e6

based on: adoptium#5195 adoptium#5167

judovana force-pushed the jtregBufferII branch from 5e13b5e to 58208e6 Compare May 10, 2024 12:12

karianna approved these changes May 14, 2024

View reviewed changes

functional/jtreg-buffer/README.md Outdated Show resolved Hide resolved

karianna reviewed May 14, 2024

View reviewed changes

Update functional/jtreg-buffer/README.md

d2ebc47

Co-authored-by: Martijn Verburg <[email protected]>

smlambert reviewed Jun 5, 2024

View reviewed changes

judovana closed this Jun 5, 2024

judovana reopened this Jun 5, 2024

karianna requested a review from smlambert June 6, 2024 00:24

judovana closed this Jul 23, 2024

smlambert mentioned this pull request Aug 1, 2024

Consider adding new test material: jtreg-buffer #5195

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added jtreg-buffer testsuite #5308

Added jtreg-buffer testsuite #5308

judovana commented May 10, 2024

judovana commented May 10, 2024

karianna left a comment

judovana commented May 14, 2024 •

edited

Loading

judovana commented May 22, 2024

sophia-guo commented May 27, 2024

smlambert commented May 27, 2024

judovana commented May 28, 2024 •

edited

Loading

smlambert Jun 5, 2024 •

edited

Loading

judovana Jun 5, 2024

smlambert commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 10, 2024 •

edited

Loading

judovana commented Jun 10, 2024

judovana commented Jun 28, 2024

judovana commented Jul 23, 2024

Added jtreg-buffer testsuite #5308

Added jtreg-buffer testsuite #5308

Conversation

judovana commented May 10, 2024

judovana commented May 10, 2024

karianna left a comment

Choose a reason for hiding this comment

judovana commented May 14, 2024 • edited Loading

judovana commented May 22, 2024

sophia-guo commented May 27, 2024

smlambert commented May 27, 2024

judovana commented May 28, 2024 • edited Loading

smlambert Jun 5, 2024 • edited Loading

Choose a reason for hiding this comment

judovana Jun 5, 2024

Choose a reason for hiding this comment

smlambert commented Jun 5, 2024 • edited Loading

judovana commented Jun 5, 2024 • edited Loading

judovana commented Jun 5, 2024 • edited Loading

judovana commented Jun 10, 2024 • edited Loading

judovana commented Jun 10, 2024

judovana commented Jun 28, 2024

judovana commented Jul 23, 2024

judovana commented May 14, 2024 •

edited

Loading

judovana commented May 28, 2024 •

edited

Loading

smlambert Jun 5, 2024 •

edited

Loading

smlambert commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 5, 2024 •

edited

Loading

judovana commented Jun 10, 2024 •

edited

Loading