Chart: Allow to extend cleanup cronjob env with custom envs from secrets or config maps

[dada-engineer]

Official Helm Chart version

1.13.1 (latest released)

Apache Airflow version

2.9.1

Kubernetes Version

1.28

Helm Chart configuration

Hi everyone,

I get some airflow config params provided via k8s secrets / configmaps (maybe from another IT department during provisioning), e.g. the remote base log folder (some s3 URI created via open tofu and provided as a k8s secret)

Currently it seems that I am not able to bring those into the cleanup cronjob because it is not rendering the extraEnv and extraEnvFrom global values, nor does it provide custom ones that are referenceable (you can only specify a name: value custom env not a valueFrom or use the standard ones built here)

As the docs state I can circumvent this by creating a custom helm chart (we actually do anyways) and then adding the cronjob I need (will also do this for now), but I think this can already be native airflow helm chart functionality

This is the job I am talking about

Docker Image customizations

Yes but I think this is unrelated to this issue

What happened

I tried to add the following env config (before checking the code actually)

cleanup:
  enabled: true
  env:
      - name: AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER
        valueFrom:
          secretKeyRef:
            name: secret-name
            key: AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER
      - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
        valueFrom:
          secretKeyRef:
            name: secret-name            key: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN

and as this is a bad format for now it crashed with the following:

helm.go:84: [debug] values don't meet the specifications of the schema(s) in the following chart(s):
airflow:
- cleanup.env.0: value is required
- cleanup.env.0: Additional property valueFrom is not allowed
- cleanup.env.1: value is required
- cleanup.env.1: Additional property valueFrom is not allowed

Which is clear when you look up how custom_container_env is built so no surprise here

What you think should happen instead

No response

How to reproduce

Try to add some custom env and reference an already existing secret or configmap.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.