Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete stale metrics on object delete #570

Merged
merged 1 commit into from
Aug 14, 2023
Merged

Delete stale metrics on object delete #570

merged 1 commit into from
Aug 14, 2023

Conversation

darkowlzz
Copy link
Contributor

@darkowlzz darkowlzz commented Aug 10, 2023

Depends on fluxcd/pkg#612

The metrics helper now accepts owned finalizers to determine if an object is no longer managed by the controller and is being deleted, and deletes the metrics associated with the object.

Call the metrics recording defer function in reconciler early to be able to record the object in deleting state.

Before this change, the following metrics continued to be exported even after the associated object is deleted for image update automation:

# HELP gotk_reconcile_condition The current condition status of a GitOps Toolkit resource reconciliation.
# TYPE gotk_reconcile_condition gauge
gotk_reconcile_condition{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",status="False",type="Ready"} 1
gotk_reconcile_condition{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",status="True",type="Ready"} 0
gotk_reconcile_condition{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",status="Unknown",type="Ready"} 0
# HELP gotk_reconcile_duration_seconds The duration in seconds of a GitOps Toolkit resource reconciliation.
# TYPE gotk_reconcile_duration_seconds histogram
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="0.01"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="0.038363583488692544"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="0.1471764538093883"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="0.5646216173286169"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="2.166090855590701"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="8.309900738254731"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="31.879757075478317"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="122.30217221643493"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="469.19495946736544"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="1799.9999999999986"} 1
gotk_reconcile_duration_seconds_bucket{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default",le="+Inf"} 1
gotk_reconcile_duration_seconds_sum{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default"} 0.009209323
gotk_reconcile_duration_seconds_count{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default"} 1
# HELP gotk_suspend_status The current suspend status of a GitOps Toolkit resource.
# TYPE gotk_suspend_status gauge
gotk_suspend_status{kind="ImageUpdateAutomation",name="test-update-auto",namespace="default"} 0

With this change, they get deleted once the associated object is deleted.

Also, the metrics helper no longer exports ConditionDelete for readiness metrics.

The metrics helper now accepts owned finalizers to determine if an
object is no longer managed by the controller and is being deleted, and
deletes the metrics associated with the object.

Call the metrics recording defer function in controller early to be able
to record the object in deleting state.

Signed-off-by: Sunny <[email protected]>
@darkowlzz darkowlzz merged commit e6570a9 into main Aug 14, 2023
7 checks passed
@darkowlzz darkowlzz deleted the stale-metrics branch August 14, 2023 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants