Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mock Tests failing in kubernetes-tests/ regarding authorization.openshift.io resources #3152

Closed
rohanKanojia opened this issue May 20, 2021 · 5 comments · Fixed by #3158 or #3157
Closed

Mock Tests failing in kubernetes-tests/ regarding authorization.openshift.io resources #3152

rohanKanojia opened this issue May 20, 2021 · 5 comments · Fixed by #3158 or #3157
Assignees
@rohanKanojia
Labels
bug
Milestone
5.5.0

Comments

@rohanKanojia
Copy link
Member

Not sure if anyone noticed, but I'm seeing consistent failures of these tests on master in kubernetes-tests/ module:

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ClusterRoleTest.delete:80 
Expecting value to be true but was false
[ERROR]   ClusterRoleTest.get:46 
Expecting actual not to be null
[ERROR]   OpenshiftRoleBindingTest.testPatchWithOnlySubjects:168 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleBindingTest.testPatchWithUserNamesAndGroupsAndNoSubjects:189 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleBindingTest.testPatchWithUserNamesAndGroupsAndOverwriteSubjects:211 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleBindingTest.testReplaceWithOnlySubjects:110 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleBindingTest.testReplaceWithUserNamesAndGroupsAndNoSubjects:128 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleBindingTest.testReplaceWithUserNamesAndGroupsAndOverwriteSubjects:148 expected: <RoleBinding(apiVersion=authorization.openshift.io/v1, groupNames=[testgroup], kind=RoleBinding, metadata=ObjectMeta(annotations=null, clusterName=null, creationTimestamp=null, deletionGracePeriodSeconds=null, deletionTimestamp=null, finalizers=[], generateName=null, generation=null, labels=null, managedFields=[], name=null, namespace=null, ownerReferences=[], resourceVersion=null, selfLink=null, uid=null, additionalProperties={}), roleRef=null, subjects=[ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser1, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=User, name=testuser2, namespace=null, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=ServiceAccount, name=svcacct, namespace=test, resourceVersion=null, uid=null, additionalProperties={}), ObjectReference(apiVersion=null, fieldPath=null, kind=Group, name=testgroup, namespace=null, resourceVersion=null, uid=null, additionalProperties={})], userNames=[testuser1, testuser2, system:serviceaccount:test:svcacct], additionalProperties={})> but was: <null>
[ERROR]   OpenshiftRoleTest.testGet:86 expected: not <null>
[ERROR]   RoleBindingRestrictionTest.delete:80 
Expecting value to be true but was false
[ERROR]   RoleBindingRestrictionTest.get:46 
Expecting actual not to be null
[ERROR] Errors: 
[ERROR]   ClusterRoleTest.list:60 » KubernetesClient Failure executing: GET at: https://...
[ERROR]   OpenshiftRoleBindingTest.testCreateInline:224 » KubernetesClient Failure execu...
[ERROR]   OpenshiftRoleBindingTest.testCreateWithOnlySubjects:47 » KubernetesClient Fail...
[ERROR]   OpenshiftRoleBindingTest.testCreateWithUserNamesAndGroupsAndNoSubjects:66 » KubernetesClient
[ERROR]   OpenshiftRoleBindingTest.testCreateWithUserNamesAndGroupsAndOverwriteSubjects:82 » KubernetesClient
[ERROR]   OpenshiftRoleTest.testList:61 » KubernetesClient Failure executing: GET at: ht...
[ERROR]   ProjectTest.testCreateProjectAndRoleBindings:192 » KubernetesClient Failure ex...
[ERROR]   RoleBindingRestrictionTest.list:60 » KubernetesClient Failure executing: GET a...
[ERROR]   SubjectAccessReviewTest.createLocalResourceAccessReview:138 » KubernetesClient
[ERROR]   SubjectAccessReviewTest.createResourceAccessReview:116 » KubernetesClient Fail...
[ERROR]   SubjectAccessReviewTest.createSelfSubjectRulesReviews:166 » KubernetesClient F...
[ERROR]   SubjectAccessReviewTest.createSubjectRulesReviews:193 » KubernetesClient Failu...
[ERROR]   SubjectAccessReviewTest.testCreate:54 » KubernetesClient Failure executing: PO...
[ERROR]   SubjectAccessReviewTest.testCreateInLine:68 » KubernetesClient Failure executi...
[ERROR]   SubjectAccessReviewTest.testCreateLocal:81 » KubernetesClient Failure executin...
[ERROR]   SubjectAccessReviewTest.testCreateLocalInLine:98 » KubernetesClient Failure ex...
[INFO] 
[ERROR] Tests run: 766, Failures: 11, Errors: 16, Skipped: 4
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10:28 min
[INFO] Finished at: 2021-05-20T21:19:07+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test (default-test) on project kubernetes-test: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/rohaan/go/src/github.com/fabric8io/kubernetes-client/kubernetes-tests/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I've even tried deleting my repository and rebuilding from scratch but I'm still seeing these failures. Surprisingly, I'm not able to see these failures on CI
rohanKanojia added a commit that referenced this issue May 20, 2021
@rohanKanojia
Fix #3152: Retry only Create-only resources in OpenShiftOAuthInterceptor
19e57dc
@rohanKanojia
Copy link
Member Author

Looks like I was getting these failures because I had my crc OpenShift cluster running locally. When I stopped the cluster, the tests started passing.

@manusa manusa reopened this May 21, 2021
@manusa
Copy link
Member

manusa commented May 21, 2021

My OpenShift tests fail whenever my ~ /.kube/config context is pointing to an OpenShift cluster, local or external.

This is really bad, and we should provide a fix, tests in kubernetes-tests module should always run in the MockServer, this is critical.

@manusa manusa added the bug label May 21, 2021
@manusa manusa added this to the 5.5.0 milestone May 21, 2021
@rohanKanojia rohanKanojia self-assigned this May 21, 2021
rohanKanojia added a commit to rohanKanojia/kubernetes-client that referenced this issue May 21, 2021
@rohanKanojia
Fix fabric8io#3152: Retry only Create-only resources in OpenShiftOAut…
466f911 
…hInterceptor

Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
@rohanKanojia rohanKanojia mentioned this issue May 21, 2021
Merged
11 tasks
@manusa manusa mentioned this issue May 21, 2021
Merged
11 tasks
rohanKanojia added a commit to rohanKanojia/kubernetes-client that referenced this issue May 21, 2021
@rohanKanojia
Fix fabric8io#3152: Retry only Create-only resources in OpenShiftOAut…
805e3f3 
…hInterceptor

Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
@manusa
Copy link
Member

manusa commented May 24, 2021

Same as #3064

rohanKanojia added a commit to rohanKanojia/kubernetes-client that referenced this issue May 24, 2021
@rohanKanojia
Fix fabric8io#3152: Retry only Create-only resources in OpenShiftOAut…
ed3f606 
…hInterceptor

Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
rohanKanojia added a commit to rohanKanojia/kubernetes-client that referenced this issue May 24, 2021
@rohanKanojia
Fix fabric8io#3152: Retry only Create-only resources in OpenShiftOAut…
6a7af04 
…hInterceptor

Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
@manusa manusa reopened this May 25, 2021
@rohanKanojia
Copy link
Member Author

Since #3158 is merged. Why did we reopen this? Or Are failures still happening o master?

rohanKanojia added a commit to rohanKanojia/kubernetes-client that referenced this issue May 25, 2021
@rohanKanojia
Fix fabric8io#3152: Retry only Create-only resources in OpenShiftOAut…
a5472f3 
…hInterceptor

Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
@manusa
Copy link
Member

manusa commented May 25, 2021

Since #3158 is merged. Why did we reopen this? Or Are failures still happening o master?

#3157

manusa pushed a commit that referenced this issue May 25, 2021
@rohanKanojia @manusa
Fix #3152: Retry only Create-only resources in OpenShiftOAuthInterceptor
0f1e7bf 
Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview
manusa pushed a commit that referenced this issue May 31, 2021
@rohanKanojia @manusa
Fix #3152: Retry only Create-only resources in OpenShiftOAuthInterceptor
86d7094 
Right now we have a check to retry request in OpenShiftOAuthInterceptor
if it's in apiGroup authorization.openshift.io/authorization.k8s.io .

This is creating problems for regular resources in this apiGroup like
Role, ClusterRole, RoleBinding, ClusterRoleBinding etc. For example,
doing create() when oAuthToken is already initialized would always be
called twice due to this retry logic. This will result in HTTP_CONFLICT
from server.

Instead of checking apiGroup, we should retry only for these Non Restful
resources in request url LocalSubjectAccessReview, LocalResourceAccessReview
ResourceAccessReview, SelfSubjectRulesReview, SubjectRulesReview, SubjectAccessReview
SelfSubjectAccessReview

(cherry picked from commit 0f1e7bf)
@skabashnyuk skabashnyuk mentioned this issue Jun 7, 2021
Merged
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rohanKanojia rohanKanojia

Labels
bug
Projects
None yet
Milestone
5.5.0
2 participants
@manusa @rohanKanojia