Argocd cannot sync itself when a healthcheck is removed from the values file

[perlinger]

Describe the bug

This bug may be related to issue #2450.
We are wondering if argocd supports making changes to health checks without restarting argocd.

Related helm chart

argo-cd

Helm chart version

v5.14.1

To Reproduce

To reproduce the issue, please follow these steps:

1. Have a cluster running with a lua health check in the values file.
2. Remove the health check.
3. We are finding that argocd starts to hang.
4. When we delete all the argocd pods and when the pods reinstall then argocd starts to sync again and the change takes affect.

Expected behavior

After removing the health check, expect argocd to sync itself

Screenshots

No response

Additional context

No response

[perlinger]

Is it the case that ArgoCD restarts on a change to the configmap named argocd-cmd-params-cm.

• Reference: https://github.com/argoproj/argo-helm/blob/main/charts/argo-cd/templates/argocd-server/deployment.yaml#L29
• Healthchecks, on the other hand, are in the configmap named argocd-cm.

[pdrastil]

@crenshaw-dev Hi Michael - if I remember right argocd-cm was watched ConfigMap with hot reload. Can you confirm that that is the correct assumption or we are just missing checksum to restart controllers on config change.

[yu-croco]

FYI
I found the same question in the past. Below is the answer.
#1664 (reply in thread)

[perlinger]

We found that adding this snippet to our values.yaml file did the trick. It basically passed the annotation right into the Deployment, which made Kubernetes restart the pod. It helped get the updated ConfigMap in place.

global:
  podAnnotations:
    "argocdcm": "argocd-cm updated 01/26/2024. This is a short term manual fix"

Thoughts?

[yu-croco]

which means Argo CD server doesn't watch some of the data in argocd-cm CM if they are updated or not .... 👀

[pdrastil]

Seems like it - as simple fix we can add another checksum for components that use that ConfigMap.

[yu-croco]

I make PR for this. 🙋