Variables are getting duplicated on upgrading airflow

[vghar-bh]

variables are getting duplicated on upgrading airflow to 2.5.3 from 2.4.1.
after upgrade when the airflow-sync-variables pod runs its not finding the existing variables and then adding it again.

airflow version 2.5.3
currently working with helm charts from Airflow Helm Chart (User Community)
https://artifacthub.io/packages/helm/airflow-helm/airflow/8.7.0

using Argocd as GitOps CD tool for Kubernetes.

logs from airflow-sync-variables:

/home/airflow/.local/lib/python3.8/site-packages/airflow/models/base.py:49 MovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9)
[�[34m2023-07-09 05:59:12,443�[0m] {�[34msync_variables.py:�[0m357} INFO�[0m - BEGIN: airflow variables sync�[0m
[�[34m2023-07-09 05:59:14,694�[0m] {�[34msync_variables.py:�[0m336} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT is missing, adding...�[0m
[�[34m2023-07-09 05:59:14,698�[0m] {�[34msync_variables.py:�[0m348} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT was successfully added.�[0m
[�[34m2023-07-09 05:59:14,774�[0m] {�[34msync_variables.py:�[0m360} INFO�[0m - END: airflow variables sync�[0m
[�[34m2023-07-09 06:00:14,864�[0m] {�[34msync_variables.py:�[0m357} INFO�[0m - BEGIN: airflow variables sync�[0m
[�[34m2023-07-09 06:00:14,990�[0m] {�[34msync_variables.py:�[0m336} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT is missing, adding...�[0m
[�[34m2023-07-09 06:00:14,994�[0m] {�[34msync_variables.py:�[0m348} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT was successfully added.�[0m
[�[34m2023-07-09 06:00:15,058�[0m] {�[34msync_variables.py:�[0m360} INFO�[0m - END: airflow variables sync�[0m
[�[34m2023-07-09 06:01:15,132�[0m] {�[34msync_variables.py:�[0m357} INFO�[0m - BEGIN: airflow variables sync�[0m
[�[34m2023-07-09 06:01:15,256�[0m] {�[34msync_variables.py:�[0m336} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT is missing, adding...�[0m
[�[34m2023-07-09 06:01:15,260�[0m] {�[34msync_variables.py:�[0m348} INFO�[0m - Variable=ENT_NTFN_CONSUMER_COUNT was successfully added.�[0m

[thesuperzapper]

@vghar-bh can you share the helm values for your variables?

Also, what is the effect of this, that is, are you seeing the correct values for the variables still?

[vghar-bh]

Hi @thesuperzapper
values.yaml :

airflow-values.txt

the values of the variables are reflecting properly but the issue is some of the variables are getting duplicated on the upgrade which are causing issues with the dags .
logs from airflow-sync-variables are provided on top.

[vghar-bh]

hi @thesuperzapper any update on this ?
Could we have a workaround to fix this