Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jdk_container_0_FAILED Error: invalid configuration, cannot specify resource limits without cgroups v2 and --cgroup-manager=systemd #19176

Open
JasonFengJ9 opened this issue Mar 18, 2024 · 5 comments

Comments

@JasonFengJ9
Copy link
Member

JasonFengJ9 commented Mar 18, 2024

Failure link

From an internal build(rhel7le-rtp-rt5-1):

java version "17.0.11-beta" 2024-04-16
IBM Semeru Runtime Certified Edition 17.0.11+6-202403161434 (build 17.0.11-beta+6-202403161434)
Eclipse OpenJ9 VM 17.0.11+6-202403161434 (build master-6b03a382a, JRE 17 Linux ppc64le-64-Bit Compressed References 20240316_625 (JIT enabled, AOT enabled)
OpenJ9   - 6b03a382a
OMR      - 1bf2ef421
JCL      - 66cf01060a based on jdk-17.0.11+6)

Rerun in Grinder - Change TARGET to run only the failed test targets.

Optional info

Failure output (captured from console output)

[2024-03-16T15:26:00.858Z] variation: NoOptions
[2024-03-16T15:26:00.858Z] JVM_OPTIONS:  -Xverbosegclog 

[2024-03-16T15:29:00.526Z] TEST: jdk/internal/platform/docker/TestDockerMemoryMetrics.java

[2024-03-16T15:29:00.527Z] STDERR:
[2024-03-16T15:29:00.527Z]  stdout: [];
[2024-03-16T15:29:00.527Z]  stderr: [Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
[2024-03-16T15:29:00.527Z] Error: invalid configuration, cannot specify resource limits without cgroups v2 and --cgroup-manager=systemd
[2024-03-16T15:29:00.527Z] ]
[2024-03-16T15:29:00.527Z]  exitValue = 125

[2024-03-16T15:31:41.650Z] TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Expected to get exit value of [0]
[2024-03-16T15:31:41.650Z] --------------------------------------------------
[2024-03-16T15:31:47.353Z] Test results: passed: 7; failed: 5
[2024-03-16T15:32:48.978Z] Report written to /home/jenkins/workspace/Test_openjdk17_j9_dev.openjdk_ppc64le_linux/aqa-tests/TKG/output_17106027597481/jdk_container_0/report/html/report.html
[2024-03-16T15:32:48.978Z] Results written to /home/jenkins/workspace/Test_openjdk17_j9_dev.openjdk_ppc64le_linux/aqa-tests/TKG/output_17106027597481/jdk_container_0/work
[2024-03-16T15:32:48.978Z] Error: Some tests failed or other problems occurred.
[2024-03-16T15:32:48.978Z] -----------------------------------
[2024-03-16T15:32:48.978Z] jdk_container_0_FAILED

50x internal grinder - #19176 (comment)

@pshipton
Copy link
Member

The testing passed on sles12le-rtp-rt1-1 and ubu20le-rtp-rt1-1. Failed on rhel7 machines (rhel7le-svl-rt5-1, rhel7le-svl-rt6-1, rhel7le-svl-rt7-1).

@llxia is this a test configuration error, should it avoid running on rhel7?

@llxia
Copy link
Contributor

llxia commented Mar 19, 2024

is this a test configuration error, should it avoid running on rhel7?

I am not aware of such a requirement for jdk_container.
The test also failed on xlinux rhel7: https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/38921/console

00:02:34.960  Exception in thread "main" java.lang.RuntimeException: Test failed! getFreeSwapSpaceSize returns -131072
00:02:34.960  	at GetFreeSwapSpaceSize.main(GetFreeSwapSpaceSize.java:54)
00:02:34.960  ];
00:02:34.960   stderr: []
00:02:34.960   exitValue = 1
00:02:34.960  
00:02:34.960  java.lang.RuntimeException: Expected to get exit value of [0]
00:02:34.960  
00:02:34.960  	at jdk.test.lib.process.OutputAnalyzer.shouldHaveExitValue(OutputAnalyzer.java:489)
00:02:34.960  	at TestGetFreeSwapSpaceSize.testGetFreeSwapSpaceSize(TestGetFreeSwapSpaceSize.java:72)
00:02:34.960  	at TestGetFreeSwapSpaceSize.main(TestGetFreeSwapSpaceSize.java:49)
00:02:34.960  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
00:02:34.960  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
00:02:34.960  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
00:02:34.960  	at java.base/java.lang.reflect.Method.invoke(Method.java:574)
00:02:34.960  	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
00:02:34.960  	at java.base/java.lang.Thread.run(Thread.java:857)
00:02:34.960  
00:02:34.960  JavaTest Message: Test threw exception: java.lang.RuntimeException
00:02:34.960  JavaTest Message: shutting down test
00:02:34.960  
00:02:34.960  
00:02:34.960  TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: Expected to get exit value of [0]
00:02:34.960  --------------------------------------------------
00:03:25.499  Test results: passed: 11; failed: 1

Unfortunately, Adoptium does not have rhel on plinux. They only have centos and ubuntu: https://ci.adoptium.net/job/Test_openjdk17_hs_dev.openjdk_ppc64le_linux/

@annaibm could you add a temporary exclude on rhel7 for now? See example: https://github.com/adoptium/TKG/blob/0c5e49d22d17f1b91e0cc054b6b8cd047497d1ac/examples/platformRequirements/playlist.xml#L100

annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 20, 2024
annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 20, 2024
related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>
annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 22, 2024
related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>
annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 22, 2024
annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 22, 2024
annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 22, 2024
@llxia
Copy link
Contributor

llxia commented Mar 25, 2024

Similar error on cent8x86-svl-rt8-1.fyre.ibm.com:
Grinder link

08:41:06  openjdk version "17.0.11-internal" 2024-04-16
08:41:06  OpenJDK Runtime Environment (build 17.0.11-internal+0-adhoc.jenkins.BuildJDK17x86-64linuxNightly)
08:41:06  Eclipse OpenJ9 VM (build master-a3f25b1faf5, JRE 17 Linux amd64-64-Bit Compressed References 20240322_708 (JIT enabled, AOT enabled)
08:41:06  OpenJ9   - a3f25b1faf5
08:41:06  OMR      - cbfcf7d9444
08:41:06  JCL      - ea48af27e19 based on jdk-17.0.11+7)
08:41:06  ];
08:41:06   stderr: [Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
08:41:06  Resource limits are not supported and ignored on cgroups V1 rootless systems
08:41:06  ]
08:41:06   exitValue = 0
08:41:06  
08:41:06  java.lang.RuntimeException: 'Memory Limit: 300.00M' missing from stdout/stderr 
08:41:06  
08:41:06  	at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:221)
08:41:06  	at TestDockerBasic.testXshowSettingsSystem(TestDockerBasic.java:71)
08:41:06  	at TestDockerBasic.main(TestDockerBasic.java:50)
08:41:06  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
08:41:06  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
08:41:06  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
08:41:06  	at java.base/java.lang.reflect.Method.invoke(Method.java:574)
08:41:06  	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
08:41:06  	at java.base/java.lang.Thread.run(Thread.java:857)
08:41:06  
08:41:06  JavaTest Message: Test threw exception: java.lang.RuntimeException
08:41:06  JavaTest Message: shutting down test
08:41:06  
08:41:06  
08:41:06  TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.RuntimeException: 'Memory Limit: 300.00M' missing from stdout/stderr
08:41:06  --------------------------------------------------
08:41:06  TEST: jdk/internal/platform/docker/TestDockerMemoryMetrics.java

Thanks for @annaibm . We also noticed the failure in cent7, sles15, ub22 and rhel8.
I suspect this is related to machine configuration.
For now, I think we should exclude the subtest: jdk/internal/platform/docker/TestDockerMemoryMetrics.java instead.

@annaibm
Copy link
Contributor

annaibm commented Mar 25, 2024

List of subtests with errors:
1.jdk/internal/platform/docker/TestDockerBasic.java.TestDockerBasic

openjdk version "17.0.11-internal" 2024-04-16
OpenJDK Runtime Environment (build 17.0.11-internal+0-adhoc.jenkins.BuildJDK17x86-64linuxNightly)
Eclipse OpenJ9 VM (build master-a3f25b1faf5, JRE 17 Linux amd64-64-Bit Compressed References 20240322_708 (JIT enabled, AOT enabled)
OpenJ9   - a3f25b1faf5
OMR      - cbfcf7d9444
JCL      - ea48af27e19 based on jdk-17.0.11+7)
];
 stderr: [Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Resource limits are not supported and ignored on cgroups V1 rootless systems
]
 exitValue = 0

java.lang.RuntimeException: 'Memory Limit: 300.00M' missing from stdout/stderr 

	at jdk.test.lib.process.OutputAnalyzer.shouldContain(OutputAnalyzer.java:221)
	at TestDockerBasic.testXshowSettingsSystem(TestDockerBasic.java:71)
	at TestDockerBasic.main(TestDockerBasic.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:574)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
	at java.base/java.lang.Thread.run(Thread.java:857)

JavaTest Message: Test threw exception: java.lang.RuntimeException
JavaTest Message: shutting down test
  1. jdk/internal/platform/docker/TestDockerMemoryMetrics.java.TestDockerMemoryMetrics
stdout: [[memory, 200m]
Exception in thread "main" java.lang.RuntimeException: Memory limit not equal, expected : [209715200], got : [-1]
	at MetricsMemoryTester.testMemoryLimit(MetricsMemoryTester.java:60)
	at MetricsMemoryTester.main(MetricsMemoryTester.java:37)
];
 stderr: [Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Resource limits are not supported and ignored on cgroups V1 rootless systems
]
 exitValue = 1

java.lang.RuntimeException: Expected to get exit value of [0]

	at jdk.test.lib.process.OutputAnalyzer.shouldHaveExitValue(OutputAnalyzer.java:489)
	at TestDockerMemoryMetrics.testMemoryLimit(TestDockerMemoryMetrics.java:106)
	at TestDockerMemoryMetrics.testMemoryLimit(TestDockerMemoryMetrics.java:90)
	at TestDockerMemoryMetrics.main(TestDockerMemoryMetrics.java:57)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:574)
	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
	at java.base/java.lang.Thread.run(Thread.java:857)

JavaTest Message: Test threw exception: java.lang.RuntimeException
JavaTest Message: shutting down test
  1. jdk/internal/platform/docker/TestGetFreeSwapSpaceSize.java.TestGetFreeSwapSpaceSize
Execution failed: `main' threw exception: java.lang.RuntimeException: Expected to get exit value of [0]    
  1. jdk/internal/platform/docker/TestLimitsUpdating.java.TestLimitsUpdating
Execution failed: `main' threw exception: java.lang.RuntimeException: 'Runtime.availableProcessors: 2' missing from stdout/stderr 
  1. jdk/internal/platform/docker/TestPidsLimit.java.TestPidsLimit
Execution failed: `main' threw exception: java.lang.RuntimeException: assertEquals: expected Unlimited to equal 1000    

@llxia Should we exclude all the 5 subtests here.

annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 25, 2024
- Exclude  jdk_container subtests for all linux versions -
`jdk/internal/platform/docker/TestDockerBasic.java`,
`jdk/internal/platform/docker/TestDockerMemoryMetrics.java`,
`jdk/internal/platform/docker/TestGetFreeSwapSpaceSize.java`,
`jdk/internal/platform/docker/TestLimitsUpdating.java`,
`jdk/internal/platform/docker/TestPidsLimit.java`

related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>
llxia pushed a commit to adoptium/aqa-tests that referenced this issue Mar 27, 2024
- Exclude  jdk_container subtests for all linux versions -
`jdk/internal/platform/docker/TestDockerBasic.java`,
`jdk/internal/platform/docker/TestDockerMemoryMetrics.java`,
`jdk/internal/platform/docker/TestGetFreeSwapSpaceSize.java`,
`jdk/internal/platform/docker/TestLimitsUpdating.java`,
`jdk/internal/platform/docker/TestPidsLimit.java`

related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>
@llxia
Copy link
Contributor

llxia commented Mar 27, 2024

subtests are excluded via adoptium/aqa-tests#5162

annaibm added a commit to annaibm/aqa-tests that referenced this issue Mar 27, 2024
- Exclude  jdk_container subtests for all linux versions -
`jdk/internal/platform/docker/TestDockerBasic.java`,
`jdk/internal/platform/docker/TestDockerMemoryMetrics.java`,
`jdk/internal/platform/docker/TestGetFreeSwapSpaceSize.java`,
`jdk/internal/platform/docker/TestLimitsUpdating.java`,
`jdk/internal/platform/docker/TestPidsLimit.java`

related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>
llxia pushed a commit to adoptium/aqa-tests that referenced this issue Mar 27, 2024
* Exclude jdk_nio,jdk_rmi,jdk_security subtests (#5182)

- Exclude `java/nio/channels/DatagramChannel/BasicMulticastTests.java` for all java versions on the Mac OS platform.
- Exclude `java/rmi/activation/Activatable/nestedActivate/NestedActivate.java` for Java 8,Java 11 versions on the Windows platform.
- Exclude `java/nio/channels/Pipe/PipeInterrupt.java` for all java versions on windows platform.
- Exclude `sun/security/ssl/SSLSocketImpl/AsyncSSLSocketClose.java` for Java 8 version on AIX platform.

related:https://github.ibm.com/runtimes/backlog/issues/1053
related:https://github.ibm.com/runtimes/backlog/issues/1004
related:eclipse-openj9/openj9#18479
related:eclipse-openj9/openj9#18575

Signed-off-by: Anna Babu Palathingal <[email protected]>

* Exclude jdk_container subtests (#5162)

- Exclude  jdk_container subtests for all linux versions -
`jdk/internal/platform/docker/TestDockerBasic.java`,
`jdk/internal/platform/docker/TestDockerMemoryMetrics.java`,
`jdk/internal/platform/docker/TestGetFreeSwapSpaceSize.java`,
`jdk/internal/platform/docker/TestLimitsUpdating.java`,
`jdk/internal/platform/docker/TestPidsLimit.java`

related: eclipse-openj9/openj9#19176

Signed-off-by: Anna Babu Palathingal <[email protected]>

---------

Signed-off-by: Anna Babu Palathingal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants