healthcheckextension: Add an ability to delay collector shutdown #34454
Labels
enhancement
New feature or request
extension/healthcheck
Health Check Extension
extension/healthcheckv2
needs triage
New item requiring triage
Component(s)
extension/healthcheck, extension/healthcheckv2
Is your feature request related to a problem? Please describe.
We have a problem with 503 errors in the K8s environment for the otel endpoint.
These errors occur during the distribution/change of the collector configuration.
We have a process to restart the collector inside the pod with the new configuration.
During the configuration rollout, the collector shuts down too quickly, so K8s doesn't update the pod's healthy state and continues sending requests to this pod, leading to 503 errors.
Describe the solution you'd like
I'd like to have a way to delay the collector/endpoint shutdown after the HC state changes to NotReady.
This way, pod readiness would catch the NotReady status and move the traffic to the other pods before the actual collector shutdown.
It could look like: yuri-rs@55e0c5a
Describe alternatives you've considered
An alternative could be a delay in the collector service itself, like (added sleep line):
However, doing it inside the healthcheck extension seems better to me.
Additional context
What do you think about this feature?
Would you accept a PR with this functionality?
The text was updated successfully, but these errors were encountered: