KerberosAuthenticationIT authentication test failures #56507
Assignees
Labels
:Security/Authentication
Logging in, Usernames/passwords, Realms (Native/LDAP/AD/SAML/PKI/etc)
Team:Security
Meta label for security team
>test-failure
Triaged test failures from CI
Comments
benwtrent
added
>test-failure
|
Pinging @elastic/es-security (:Security/Authentication) |
|
These started failing in May 9 only with runtime java set to 15 ( this is why @benwtrent can't reproduce them I guess, the repro line above is missing the |
|
Here is a different test failure, but also JDK15 + kerberos NPE: https://gradle-enterprise.elastic.co/s/hgrh76kyapze2 |
|
Another JDK15 + Kerberos NPE today https://gradle-enterprise.elastic.co/s/w46qi55v5ga2k |
|
I spent 30 mins on it today out of curiosity. It seems this is actually a bug in sun.security.krb5.KrbKdcRep that was surfaced by the changes to |
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this issue
May 14, 2020
Somewhat convoluted AwaitsFix for elastic#56507 that only applies on JDK15.
DaveCTurner
added a commit
that referenced
this issue
May 15, 2020
Somewhat convoluted AwaitsFix for #56507 that only applies on JDK15.
DaveCTurner
added a commit
that referenced
this issue
May 15, 2020
Somewhat convoluted AwaitsFix for #56507 that only applies on JDK15.
DaveCTurner
added a commit
that referenced
this issue
May 15, 2020
Somewhat convoluted AwaitsFix for #56507 that only applies on JDK15.
DaveCTurner
added a commit
that referenced
this issue
May 15, 2020
Somewhat convoluted AwaitsFix for #56507 that only applies on JDK15.
|
I believe this same issue is causing failures on the JVM matrix builds, as the HDFS fixture depends on Kerberos and also hits this same failure. Sample build scans: All on JDK15. The error appears to be the same: Very log failure log |
|
So this continues to fail on the JDK 15 builds. What should we do here? Should we disable this fixture and associated tests on Java 15? I'm ok with doing so temporarily but we should put some level of effort into figuring out if there's some kind of Java 15 bug given our trend of moving to the latest JDK on release and Java 15 GA only three months away. Here's the last week's failures for this issue: |
|
I'll get to it in the next couple of days ,it's on my radar |
|
This is likely caused by https://bugs.openjdk.java.net/browse/JDK-8246193, which should be fixed in the next JDK 15 build |
|
Thanks @jaymode , yes this raised bug agrees with what I saw in my initial analysis too. Happy our waiting game paid off :) |
|
I'm seeing several similar looking failures today, seems to have started some time around yesterday on 7.8 to 7.x branches. 7.8: https://gradle-enterprise.elastic.co/s/u3obnnvb4b2a2 (ES_RUNTIME_JAVA=zulu8) There are some from yesterday (20.7) that uses ES_RUNTIME_JAVA=openjdk15 as well: I wonder if these builds were added later. There seems to be an uptic starting yesterday. Also I wonder why #56767 doesn't seem to work to supress the last two failures on JDK 15. |
|
This is probably related too: https://gradle-enterprise.elastic.co/s/xy7b6zqcjegva/console-log?task=:plugins:repository-hdfs:secureHdfsFixture |
|
The Zulu and JDK 15 builds have been around for some time now. It's possible a recent update to those JDKs is causing issues. JDK 15 especially since we are on EA builds get's updated regularly. |
|
Another instance of this: https://gradle-enterprise.elastic.co/s/fh2hmhsjyc6ii/tests/failed |
|
Another instance: https://gradle-enterprise.elastic.co/s/b2phxw4foj3xg/console-log/raw |
These two failures are actually from May 15 and they are both before the suppress (#56767). The new failures are all about zulu8, which I am still digging. |
|
We've previously seen that Zulu is faster at releasing builds with backported changes than other OpenJDK providers, so it may be that they've merged whatever broke in JDK15. If so, that sounds bad, and we need to get on top of it quickly. |
This is true. This is caused by the same bug that @jaymode linked before, i.e. https://bugs.openjdk.java.net/browse/JDK-8246193 The bug affects openjdk Also the bug is fixed in the latest JDK 15 builds (b26). I tested and it worked locally. So these tests can be unmuted for JDK 15 since we are already using the latest JDK15 b33. |
ywangd
added a commit
that referenced
this issue
Jul 28, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. Relates: #56507
|
The tests are now unmuted for jdk 15 by #60260 |
ywangd
added a commit
to ywangd/elasticsearch
that referenced
this issue
Jul 28, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. Relates: elastic#56507
|
Just had a failure on Zulu8 / 7.9. Unsure if that is expected or not given the recent muting: |
Sorry you encountered it again. The mute for zulu8 has not been merged yet. Will action soon. |
ywangd
added a commit
that referenced
this issue
Jul 29, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: #56507
ywangd
added a commit
to ywangd/elasticsearch
that referenced
this issue
Jul 29, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: elastic#56507
This was referenced Jul 29, 2020
ywangd
added a commit
to ywangd/elasticsearch
that referenced
this issue
Jul 29, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: elastic#56507
ywangd
added a commit
that referenced
this issue
Jul 29, 2020
) The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: #56507
ywangd
added a commit
that referenced
this issue
Jul 29, 2020
) The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: #56507
ywangd
added a commit
that referenced
this issue
Jul 29, 2020
The JDK bug (https://bugs.openjdk.java.net/browse/JDK-8246193) is fixed since b26. The tests can be unmuted since we are already using b33. However the same bug is now affecting jdk 8u262, which is the base for current Zulu jdk 8.48. This PR mute the tests for this specific jdk version. Relates: #56507
|
Backports all done. Since the mute is for a specific jdk version, there is no need to unmute it. Future JDK versions will automatically get picked up. Hence I am closing this issue. |
|
This just failed in a build that was using Java The build was https://gradle-enterprise.elastic.co/s/y5n7s7aijsy6e Looking in the console log it seems that the JDK was It looks like more JDK versions are affected. |
|
https://gradle-enterprise.elastic.co/s/t4epntiapfnfm is the same - also using |
|
Just failed again on zulu8: https://gradle-enterprise.elastic.co/s/swcbiydrrw2ce |
|
Argh, zulu released
I'll re-mute so that it skips all versions of [262, 271) to be sure. Sorry for the noise. |
|
I raised #60995 to mute it for any versions falls in between 8u[262, 271) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Build scan:
7.6 https://gradle-enterprise.elastic.co/s/3olzbtxfidvds
7.8 https://gradle-enterprise.elastic.co/s/22optb3r7tq24
7.7 https://gradle-enterprise.elastic.co/s/4225d2fztqtfg
Repro line:
Three tests fail:
Reproduces locally?:
No
Applicable branches:
7.6
7.7
7.8
master
Failure excerpt:
All tests fail with a similar NPE.
The text was updated successfully, but these errors were encountered: