-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Enable automatic cleanup of finished compaction jobs and make TTL configurable #556
Comments
/assign |
#711 introduced functionality which deletes all compaction jobs - after they enter the etcd-druid/controllers/compaction/reconciler.go Lines 140 to 171 in fec5c09
The controller reconciles whenever there is a job status change event, or when the snapshot lease gets updated. These events would make the controller:
Even for a job which fails when Halting work on this for now, until further brainstorming reveals edge cases which might not be covered in #711. |
We will need to relook at the compaction controller design and re-think garbage collection of completed compaction jobs. |
Feature (What you would like to be added):
Druid creates compaction jobs for every etcd cluster based on crossing a defined threshold of accumulated events since the last compaction run. These jobs run to completion but are not automatically removed. An operator today has to manually delete these jobs.
Since k8s version 1.21 spec.ttlSecondsAfterFinished can be set to configure after how much time the job resource should be removed.
Introduce a new flag that the consumer can pass onto druid to configure the value for the TTL. Assume a default value (> 0) of
ttlSecondsAfterFinished
and use it if none is explicitly passed by the consumer for all compaction jobs across all etcd clusters.Motivation (Why is this needed?):
To prevent additional work for an operator to manually remove completed jobs.
The text was updated successfully, but these errors were encountered: