Application status is always Progressing when programmatically instantiated Pods are present

[cmpilato]

I'm a confessing Kubernetes/ArgoCD newbie, asking what I hope is not something I managed to overlook on page 1 of the docs, but...

My application is a mixed bag in terms of k8s resource declaration. Most of the application is explicitly defined by k8s YAML files (Ingresses, Deployments, Services, etc.). But a portion of the application is dynamic, Pods created by code running in the application itself from PodTemplates defined in those same k8s YAML files. It is this aspect of the application that I believe is causing my application to constantly appear in a "Progressing" state in ArgoCD.

Are there any best practices to be aware of as regards ArgoCD application health determination in such a scenario?

[crenshaw-dev]

I'm a confessing Kubernetes/ArgoCD newbie

Welcome aboard, we're all newbies here in one way or another! 😄

Can you post a screenshot of your app's resource tree in the Argo CD UI? I'd be surprised if a programmatically-generated Pod were keeping the app stuck in Progressing. Some top-level resource (i.e. some resource defined in git) must be Progressing, so we just gotta find that one and determine why it's not going Healthy.

[crenshaw-dev]

I think the tool which is spinning up these Pods must be adding the Argo CD tracking label and/or annotation. That's causing Argo CD to think, "ah, this used to be a resource represented in git, but now it's been removed from git." That's why the Pod is marked as "requires pruning."

You have a couple options. Either:

1. change the programmatic Pod-spinny-uppy tool to not attach the Argo CD tracking label and/or annotation to the Pod
2. explicitly mark the Pod as extraneous by adding an annotation

[cmpilato]

So long as I can find a way to work the term "Pod-spinny-uppy tool" into our documentation, I'll give these options a try. Thanks!

[cmpilato]

Hrm... The following is already present on these Pods:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    argocd.argoproj.io/compare-options: IgnoreExtraneous
    argocd.argoproj.io/sync-options: Prune=false
...

[brhodes-pcty]

Hi @cmpilato -- did you ever find a solution for this? We're dealing with a similar challenge at the moment. Everything works, but the listener pod just sits at progressing, forever.

[cmpilato]

Unfortunately, I did not.