Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

openjdk net related tests failed on test-skytap-ubuntu2004-ppc64le-1 #2355

Closed
sophia-guo opened this issue Oct 14, 2021 · 15 comments
Closed

openjdk net related tests failed on test-skytap-ubuntu2004-ppc64le-1 #2355

sophia-guo opened this issue Oct 14, 2021 · 15 comments
Assignees
@Haroon-Khel
Labels
arch:ppc64le (pLinux) testFail

Comments

@sophia-guo
Copy link

75 openjdk tests failed on test-skytap-ubuntu2004-ppc64le-1. All tests are related with net connections: remote, net, connection, socket related. Tests running on this machine also takes triple times. Though it might be the failures make the running time longer than normal.

The whole set is:
jdk/jfr/jmx/TestGetRecordings.java.TestGetRecordings
javax/management/remote/mandatory/connection/RMIConnectionIdTest.java.RMIConnectionIdTest
javax/management/remote/mandatory/notif/RMINotifTest.java.RMINotifTest
javax/management/remote/mandatory/passwordAuthenticator/RMIAltAuthTest.java.RMIAltAuthTest
javax/management/remote/mandatory/passwordAuthenticator/RMIPasswdAuthTest.java.RMIPasswdAuthTest
javax/management/remote/mandatory/socketFactories/RMISocketFactoriesTest.java.RMISocketFactoriesTest
javax/management/remote/mandatory/subjectDelegation/SubjectDelegation1Test.java.SubjectDelegation1Test
javax/management/remote/mandatory/util/MapNullValuesTest.java.MapNullValuesTest
sun/management/jmxremote/bootstrap/CustomLauncherTest.java.CustomLauncherTest
sun/management/jmxremote/bootstrap/JMXInterfaceBindingTest.java.JMXInterfaceBindingTest
sun/management/jmxremote/bootstrap/JvmstatCountersTest.java.JvmstatCountersTest
sun/management/jmxremote/bootstrap/LocalManagementTest.java.LocalManagementTest
java/net/CookieHandler/CookieManagerTest.java.CookieManagerTest
java/net/DatagramPacket/ReuseBuf.java.ReuseBuf
java/net/DatagramSocket/GetLocalAddress.java.GetLocalAddress
java/net/DatagramSocket/PortUnreachable.java.PortUnreachable
java/net/DatagramSocket/ReuseAddressTest.java.ReuseAddressTest
java/net/DatagramSocket/Send12k.java.Send12k
java/net/DatagramSocket/SendReceiveMaxSize.java.SendReceiveMaxSize
java/net/DatagramSocket/SendSize.java.SendSize
java/net/DatagramSocket/SetDatagramSocketImplFactory/ADatagramSocket.java.ADatagramSocket
java/net/MulticastSocket/B6427403.java.B6427403
java/net/MulticastSocket/IPMulticastIF.java.IPMulticastIF
java/net/ServerSocket/TestLocalAddress.java.TestLocalAddress
java/net/Socket/CloseAvailable.java.CloseAvailable
java/net/Socket/GetLocalAddress.java.GetLocalAddress
java/net/Socket/InheritTimeout.java.InheritTimeout
java/net/Socket/ProxyCons.java.ProxyCons
java/net/Socket/ReadTimeout.java.ReadTimeout
java/net/Socket/SetSoLinger.java.SetSoLinger
java/net/Socket/SoTimeout.java.SoTimeout
java/net/Socket/asyncClose/AsyncClose.java.AsyncClose
java/net/Socket/asyncClose/BrokenPipe.java.BrokenPipe
java/net/Socket/setReuseAddress/Basic.java.Basic
java/net/Socket/setReuseAddress/Restart.java.Restart
java/net/SocketInputStream/SocketTimeout.java.SocketTimeout
java/net/Socks/BadProxySelector.java.BadProxySelector
java/net/Socks/SocksProxyVersion.java.SocksProxyVersion
jdk/net/ExtendedSocketOption/AsynchronousSocketChannelNAPITest.java.AsynchronousSocketChannelNAPITest
jdk/net/ExtendedSocketOption/DatagramChannelNAPITest.java.DatagramChannelNAPITest
jdk/net/ExtendedSocketOption/DatagramSocketNAPITest.java.DatagramSocketNAPITest
jdk/net/ExtendedSocketOption/SocketChannelNAPITest.java.SocketChannelNAPITest
jdk/net/ExtendedSocketOption/SocketNAPITest.java.SocketNAPITest
sun/net/www/http/HttpClient/B8025710.java.B8025710
sun/net/www/http/KeepAliveCache/B5045306.java.B5045306
sun/net/www/protocol/http/B6299712.java.B6299712
sun/net/www/protocol/http/HttpOnly.java.HttpOnly
sun/net/www/protocol/http/NoCache.java.NoCache
sun/net/www/protocol/http/RelativeRedirect.java.RelativeRedirect
sun/net/www/protocol/http/SetChunkedStreamingMode.java.SetChunkedStreamingMode
sun/net/www/protocol/https/HttpsURLConnection/B6226610.java.B6226610
sun/net/www/protocol/https/HttpsURLConnection/ImpactOnSNI.java.ImpactOnSNI
sun/net/www/protocol/https/HttpsURLConnection/PostThruProxyWithAuth.java.PostThruProxyWithAuth
java/nio/channels/DatagramChannel/AdaptorBasic.java.AdaptorBasic
java/nio/channels/DatagramChannel/EmptyBuffer.java.EmptyBuffer
java/nio/channels/DatagramChannel/IsBound.java.IsBound
java/nio/channels/DatagramChannel/IsConnected.java.IsConnected
java/nio/channels/DatagramChannel/Sender.java.Sender
java/nio/channels/Selector/Alias.java.Alias
java/nio/channels/Selector/BasicConnect.java.BasicConnect
java/nio/channels/Selector/Connect.java.Connect
java/nio/channels/Selector/ConnectWrite.java.ConnectWrite
java/nio/channels/Selector/KeysReady.java.KeysReady
java/nio/channels/Selector/SelectorLimit.java.SelectorLimit
java/nio/channels/ServerSocketChannel/Basic.java.Basic
java/nio/channels/ServerSocketChannel/NonBlockingAccept.java.NonBlockingAccept
java/nio/channels/SocketChannel/AdaptSocket.java.AdaptSocket
java/nio/channels/SocketChannel/Basic.java.Basic
java/nio/channels/SocketChannel/Connect.java.Connect
java/nio/channels/SocketChannel/ConnectState.java.ConnectState
java/nio/channels/SocketChannel/FinishConnect.java.FinishConnect
java/nio/channels/SocketChannel/Hangup.java.Hangup
java/nio/channels/SocketChannel/LingerOnClose.java.LingerOnClose
java/nio/channels/SocketChannel/LocalAddress.java.LocalAddress
java/nio/channels/SocketChannel/VectorParams.java.VectorParams

Rerun one test on different machine test-osuosl-centos74-ppc64le-1 looks good:
https://ci.adoptopenjdk.net/view/work-in-progress/job/grinder_sandbox_new/351/
rerun on test-skytap-ubuntu2004-ppc64le-1 failed
https://ci.adoptopenjdk.net/view/work-in-progress/job/grinder_sandbox_new/350/
@sophia-guo sophia-guo mentioned this issue Oct 14, 2021
Closed
@sxa
Copy link
Member

sxa commented Oct 14, 2021

Can you summarise the underlying problems you're seeing - are they timeouts, functional issues, related to external host lookups etc?

@sophia-guo
Copy link
Author

Updated with job link:
https://ci.adoptopenjdk.net/job/Test_openjdk17_hs_extended.openjdk_ppc64le_linux_testList_1/15/

Different failures, but looks like similar related with net address, or connections. For example:
Execution failed: `main' threw exception: java.lang.RuntimeException: Cannot assign requested address

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Oct 15, 2021 •
edited
Loading

I think the issue is caused by the machine, test-skytap-ubuntu2004-ppc64le-1, not being able to detect its external ipv4 address or that it thinks it's external ipv4 address does not belong to the machine.

If I run the debug role on both machines, test-osuosl-centos74-ppc64le-1 being the working machine, I get

"ansible_default_ipv4.address: 140.211.168.228",

for test-osuosl-centos74-ppc64le-1 and

"ansible_default_ipv4.address: 10.0.0.1",

for test-skytap-ubuntu2004-ppc64le-1, when it should have found its own ipv4 address. We had a similar issue on scaleway machine I believe, #1994.

I think this is caused by the network interface config for both machines:

test-osuosl-centos74-ppc64le-1:

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 140.211.168.228

test-skytap-ubuntu2004-ppc64le-1:

ibmveth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.0.1

@Haroon-Khel
Copy link
Contributor

I think changing the inet on the ibmveth3 interface to the public ipv4 will resolve the issues, but I have done something like this in the past before (on a different machine) and it immediately booted me out of the machine since I was connected remotely. I'll try to find another way

@sxa
Copy link
Member

sxa commented Oct 18, 2021

The question at this point is what problems does it cause? Both addresses should be "valid" for the machine, so it's a matter of seeing what specifically is causing the test failures. We probably need to look into what the tests are doing and whether there is something "wrong" or non-functional with the interface that has the private (10.) address? The fact it's a provate IP address shouldn't inherently causet the tests to fail. But as you say, please don't try to reconfigure that IP as it won't be able to route out to the internet any more as it won't find it's way out of its private network :-)

@sxa sxa mentioned this issue Oct 19, 2021
Closed
@sophia-guo
Copy link
Author

Could we take this machine offline (or disabled for test jobs, like, remove ci.role.test label). So without rerun we could know if those 75 tests could have some other hidden issues.

@Haroon-Khel
Copy link
Contributor

More net related tests are failing on this machine https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/1838/testReport/
jdk_nio_1,jdk_net_1,jdk_management_1,jdk_jfr_0,jdk_jmx_1 part of the hotspot extended openjdk test suite

@Haroon-Khel Haroon-Khel mentioned this issue Oct 19, 2021
Closed
@smlambert
Copy link
Contributor

+1 to take this one offline (or remove ci.role.test label) to make it clearer for triaging during release.

@Haroon-Khel
Copy link
Contributor

Ive removed the ci.role.test label from the machine for now

@sxa
Copy link
Member

sxa commented Nov 9, 2021

Since we have now completed the release on this platform we should re-enable this and try and get a solution.

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Nov 16, 2021 •
edited
Loading

ref #2358 (comment)
I've added 10.0.0.1 to the machine's /etc/hosts file
Rerunning an extended openjdk grinder https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/2551/console

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Nov 17, 2021 •
edited
Loading 

jdk_other_0
jdk_other_1
jdk_other_2
jdk_net_0
jdk_net_1
jdk_security1_0
jdk_security1_1
jdk_security2_0
jdk_security2_1
jdk_security4_0
jdk_security4_1
jdk_jdi_0
jdk_jdi_1

all failed in the above extended openjdk grinder

I've run them again in https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/2552/console after tweaking the /etc/hosts file again and now only

java/net/httpclient/websocket/WebSocketServerDriver.java.WebSocketServerDriver

from jdk_net_0, jdk_net_1 (I think it repeated the test in both)

11:25:03  cd /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/scratch/0 && \
11:25:03  DISPLAY=:0 \
11:25:03  HOME=/home/jenkins \
11:25:03  LANG=en_US.UTF-8 \
11:25:03  PATH=/bin:/usr/bin:/usr/sbin \
11:25:03      /home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/bin/javac \
11:25:03          -J-ea \
11:25:03          -J-esa \
11:25:03          -J-Xmx512m \
11:25:03          -J-XX:-UseCompressedOops \
11:25:03          -J-Djava.library.path=/home/jenkins/workspace/Grinder/openjdkbinary/openjdk-test-image/jdk/jtreg/native \
11:25:03          -J-Dtest.vm.opts='-ea -esa -Xmx512m -XX:-UseCompressedOops' \
11:25:03          -J-Dtest.tool.vm.opts='-J-ea -J-esa -J-Xmx512m -J-XX:-UseCompressedOops' \
11:25:03          -J-Dtest.compiler.opts= \
11:25:03          -J-Dtest.java.opts= \
11:25:03          -J-Dtest.jdk=/home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image \
11:25:03          -J-Dcompile.jdk=/home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image \
11:25:03          -J-Dtest.timeout.factor=8.0 \
11:25:03          -J-Dtest.nativepath=/home/jenkins/workspace/Grinder/openjdkbinary/openjdk-test-image/jdk/jtreg/native \
11:25:03          -J-Dtest.root=/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk \
11:25:03          -J-Dtest.name=java/net/httpclient/websocket/WebSocketServerDriver.java \
11:25:03          -J-Dtest.file=/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/WebSocketServerDriver.java \
11:25:03          -J-Dtest.src=/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket \
11:25:03          -J-Dtest.src.path=/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket \
11:25:03          -J-Dtest.classes=/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d \
11:25:03          -J-Dtest.class.path=/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d \
11:25:03          -J-Dtest.class.path.prefix=/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d:/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket \
11:25:03          -J-Dtest.modules='java.net.http/jdk.internal.net.http.websocket:open jdk.httpserver' \
11:25:03          -J-Dtest.patch.path= \
11:25:03          --add-modules java.net.http,jdk.httpserver \
11:25:03          -d /home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d/patches/java.net.http \
11:25:03          --patch-module java.net.http=/home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/java.net.http:/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d/patches/java.net.http \
11:25:03          -classpath /home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/java.net.http:/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16371456399041/jdk_net_1/work/classes/1/java/net/httpclient/websocket/WebSocketServerDriver.d \
11:25:03          --add-reads java.net.http=ALL-UNNAMED /home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/java.net.http/jdk/internal/net/http/websocket/WebSocketAndHttpTest.java
11:25:03  direct:
11:25:03  /home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/java.net.http/jdk/internal/net/http/websocket/WebSocketResponder.java:46: error: constructor MessageDecoder in class MessageDecoder cannot be applied to given types;
11:25:03          this.decoder = new MessageDecoder(consumer, true);
11:25:03                         ^
11:25:03    required: MessageStreamConsumer
11:25:03    found:    MessageStreamConsumer,boolean
11:25:03    reason: actual and formal argument lists differ in length
11:25:03  /home/jenkins/workspace/Grinder/aqa-tests/openjdk/openjdk-jdk/test/jdk/java/net/httpclient/websocket/java.net.http/jdk/internal/net/http/websocket/WebSocketResponder.java:47: error: constructor MessageEncoder in class MessageEncoder cannot be applied to given types;
11:25:03          this.encoder = new MessageEncoder(true);
11:25:03                         ^
11:25:03    required: no arguments
11:25:03    found:    boolean
11:25:03    reason: actual and formal argument lists differ in length
11:25:03  2 errors
11:25:03  
11:25:03  TEST RESULT: Failed. Compilation failed: Compilation failed

@Haroon-Khel
Copy link
Contributor

The failing tests now pass. https://ci.adoptopenjdk.net/view/Test_grinder/job/Grinder/2553/console

Ive reenabled the machine for testing

@sophia-guo
Copy link
Author

jdk_other com/sun/jndi/ldap/LdapPoolTimeoutTest.java failed on test-skytap-ubuntu2004-ppc64le-1 passed on others https://trss.adoptium.net/deepHistory?testId=62b783ac250c3c428c3aaebc

@sophia-guo sophia-guo reopened this Jun 27, 2022
@sophia-guo sophia-guo mentioned this issue Jun 27, 2022
Closed
@sxa sxa mentioned this issue Feb 6, 2023
Open
@sxa
Copy link
Member

sxa commented Feb 6, 2023

jdk_other com/sun/jndi/ldap/LdapPoolTimeoutTest.java failed on test-skytap-ubuntu2004-ppc64le-1 passed on others https://trss.adoptium.net/deepHistory?testId=62b783ac250c3c428c3aaebc

Closing this out as I've created a separate issue for it: #2924

@sxa sxa closed this as completed Feb 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Haroon-Khel Haroon-Khel

Labels
arch:ppc64le (pLinux) testFail
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@smlambert @sxa @sophia-guo @Haroon-Khel