Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix #1128: Removed multiple DPA get calls for Reconcilers #1316

Merged
merged 1 commit into from
Oct 17, 2024
Merged

fix #1128: Removed multiple DPA get calls for Reconcilers #1316

merged 1 commit into from
Oct 17, 2024

Conversation

stillalearner
Copy link
Contributor

Fixed #1128

kaovilai
kaovilai requested changes Feb 6, 2024
View reviewed changes
controllers/bsl.go Outdated Show resolved Hide resolved
kaovilai
kaovilai previously approved these changes Feb 7, 2024
View reviewed changes
Copy link
Member

@kaovilai kaovilai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@kaovilai

This comment was marked as outdated.

Sign in to view
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 7, 2024
@stillalearner

This comment was marked as outdated.

Sign in to view
3 similar comments
@stillalearner

This comment was marked as outdated.

Sign in to view
@stillalearner

This comment was marked as outdated.

Sign in to view
@stillalearner

This comment was marked as outdated.

Sign in to view
@weshayutin
Copy link
Contributor

@sseago @shubham-pampattiwar please review :)

@shubham-pampattiwar
Copy link
Member

shubham-pampattiwar commented Feb 12, 2024 •
edited
Loading

I am unsure about the changes here, I think the rationale behind fetching the DPA explicitly in every reconcile function was to get the latest DPA available in the cluster.

@stillalearner
Copy link
Contributor Author

I am unsure about the changes here, I think the rationale behind fetching the DPA explicitly in every reconcile function was to get the latest DPA available in the cluster.

But we are not updating the DPA after validation right? so we can use the same DPA fetching it once?

@kaovilai
Copy link
Member

kaovilai commented Feb 12, 2024 •
edited
Loading

I am unsure about the changes here, I think the rationale behind fetching the DPA explicitly in every reconcile function was to get the latest DPA available in the cluster.

Each reconcile you see is called right after the other in the same loop. Current implementation prior to this PR mean we're doing like 10 or so get calls within a span of 1-2 seconds, more likely to have issues with "outdated-copy" during update calls that rely on multiple gets so close to each other. Also the client backoff throttling.

I think it's best to get once, update once in entire batch.

Perhaps add to the end func ReconcileUpdateDPA() to do the update/patch call once.

This would help in cu environments with busy api server.

@kaovilai
Copy link
Member 

	if err := r.Get(ctx, req.NamespacedName, &dpa); err != nil {
		log.Error(err, "unable to fetch DataProtectionApplication CR")
		return result, nil
	}
+	//--deep copy here

	// set client to pkg/client for use in non-reconcile functions
	oadpClient.SetClient(r.Client)

	_, err := ReconcileBatch(r.Log,
		r.ValidateDataProtectionCR,
		r.ReconcileFsRestoreHelperConfig,
...
	)

	if err != nil {
		apimeta.SetStatusCondition(&dpa.Status.Conditions,
			metav1.Condition{
				Type:    oadpv1alpha1.ConditionReconciled,
				Status:  metav1.ConditionFalse,
				Reason:  oadpv1alpha1.ReconciledReasonError,
				Message: err.Error(),
			},
		)

	} else {
		apimeta.SetStatusCondition(&dpa.Status.Conditions,
			metav1.Condition{
				Type:    oadpv1alpha1.ConditionReconciled,
				Status:  metav1.ConditionTrue,
				Reason:  oadpv1alpha1.ReconciledReasonComplete,
				Message: oadpv1alpha1.ReconcileCompleteMessage,
			},
		)
	}
+	// patch spec updates
	statusErr := r.Client.Status().Update(ctx, &dpa)
	if err == nil { // Don't mask previous error
		err = statusErr
	}

@stillalearner
Copy link
Contributor Author

@kaovilai why do we need a deep copy here? Do you mean to create a copy of DPA (lets say copiedDpa) and pass that copiedDpa in all the reconcile functions, and then based on that copiedDpa, update the dpa using Patch() ?

@kaovilai
Copy link
Member

kaovilai commented Feb 13, 2024 •
edited
Loading

I meant deepcopy to have orignal in the function, then pass to copy to funcs, that way you have orignalFromClusterDPA, and updatedFromReconcilerDpa

Example: r.Client.Patch(ctx, newDeploy, client.MergeFrom(original))

@kaovilai
Copy link
Member

Reconcile shouldn't update dpa.. so ignore my comment about patching.

@stillalearner
Copy link
Contributor Author

@shubham-pampattiwar review please.

@openshift-ci openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 14, 2024
@kaovilai

This comment was marked as outdated.

Sign in to view
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 11, 2024
@openshift-ci openshift-ci bot removed the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 11, 2024
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 11, 2024
@stillalearner
Copy link
Contributor Author

/retest

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 11, 2024
kaovilai
kaovilai reviewed Oct 14, 2024
View reviewed changes
controllers/dpa_controller.go
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @shubham-pampattiwar
Please check if this change is ok with you.

@shubham-pampattiwar
Copy link
Member

@stillalearner Please update the PR to fix #1127 as well, changes are similar and less disruptive. Or Do you want to create a separate PR for it ?

@stillalearner
Copy link
Contributor Author

@stillalearner Please update the PR to fix #1127 as well, changes are similar and less disruptive. Or Do you want to create a separate PR for it ?

@shubham-pampattiwar Separate would be better for 1127 as i have to still go through that one.. Lets close this one first.

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 15, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kaovilai, mateusoliveira43, shubham-pampattiwar, stillalearner

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [kaovilai,mateusoliveira43,shubham-pampattiwar]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kaovilai
Copy link
Member

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 15, 2024
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD e5898f4 and 2 for PR HEAD ee2f611 in total

1 similar comment
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD e5898f4 and 2 for PR HEAD ee2f611 in total

@kaovilai
Copy link
Member

/retest-required

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD e5898f4 and 2 for PR HEAD ee2f611 in total

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 65ae702 and 1 for PR HEAD ee2f611 in total

@kaovilai
Copy link
Member

infra flake

error: build error: creating build container: initializing source docker://quay.io/konveyor/builder:latest: reading manifest latest in quay.io/konveyor/builder: received unexpected HTTP status: 502 Bad Gateway

/override ci/prow/4.17-e2e-test-kubevirt-aws

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 16, 2024

@kaovilai: Overrode contexts on behalf of kaovilai: ci/prow/4.17-e2e-test-kubevirt-aws

In response to this:

infra flake

error: build error: creating build container: initializing source docker://quay.io/konveyor/builder:latest: reading manifest latest in quay.io/konveyor/builder: received unexpected HTTP status: 502 Bad Gateway

/override ci/prow/4.17-e2e-test-kubevirt-aws

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Oct 17, 2024 •
edited
Loading

@stillalearner: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/4.13-e2e-test-azure a595dc2 link true /test 4.13-e2e-test-azure
ci/prow/4.12-e2e-test-aws a595dc2 link true /test 4.12-e2e-test-aws
ci/prow/4.13-e2e-test-aws a595dc2 link true /test 4.13-e2e-test-aws
ci/prow/4.14-e2e-test-azure a595dc2 link true /test 4.14-e2e-test-azure
ci/prow/4.12-e2e-test-azure a595dc2 link true /test 4.12-e2e-test-azure
ci/prow/4.18-dev-preview-e2e-test-aws ee2f611 link false /test 4.18-dev-preview-e2e-test-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD 65ae702 and 2 for PR HEAD ee2f611 in total

@openshift-merge-bot openshift-merge-bot bot merged commit 00155e1 into openshift:master Oct 17, 2024
21 of 22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaovilai kaovilai kaovilai approved these changes

@mateusoliveira43 mateusoliveira43 mateusoliveira43 approved these changes

@shubham-pampattiwar shubham-pampattiwar shubham-pampattiwar approved these changes

@savitharaghunathan savitharaghunathan Awaiting requested review from savitharaghunathan

Assignees

@kaovilai kaovilai

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Right now each reconcile functions call get for DPA, we can do it once and pass it to each function
8 participants
@stillalearner @kaovilai @weshayutin @shubham-pampattiwar @mateusoliveira43 @openshift-bot @openshift-ci-robot @openshift-merge-robot