All Airflow Configurations set via Environment Variable are masked when expose_config is set as non-sensitive-only

[wolfier]

Apache Airflow version

2.5.0

What happened

In Airflow 2.4.0, a new feature was added that added an option to mask sensitive data in UI configuration page (PR). I have set AIRFLOW__WEBSERVER__EXPOSE_CONFIG as NON-SENSITIVE-ONLY.

The feature is working partially as the airflow.cfg file display only has sensitive configurations marked as < hidden >. However, the Running Configuration table below the file display has all configuration set via environment variables marked as < hidden > which I believe is unintended.

I did not change airflow.cfg so the value here is displaying the default value of False as expected.

The value for expose_config I expect to be shown as NON-SENSITIVE-ONLY but it shown as < hidden >.

What you think should happen instead

As mentioned previously, the value for expose_config I expect to be shown as NON-SENSITIVE-ONLY.

Only the sensitive variables should be set as < hidden >.

How to reproduce

Set an Airflow configuration through the environment variable and check on the Configuration page.

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wolfier]

I believe the issue is because when the configuration is fetched as a dictionary, the values have already been hidden because display_sensitive is passed as False. The dictionary is built by calling _include_envs which hides any configuration that does not end with _cmd and _secret.

Instead, conf.as_dict should be passed with display_sensitive as True. That way, all configurations are not hidden and can be selectively hidden with SENSITIVE_CONFIG_VALUES.

conf_dict = conf.as_dict(True, True)

for sect, key in SENSITIVE_CONFIG_VALUES:
    if sect in conf_dict and key in conf_dict[sect]:
        value, source = conf_dict[sect][key]
        conf_dict[sect][key] = ("< hidden >", source)

table = [
    (section, key, str(value), source)
    for section, parameters in conf_dict.items()
    for key, (value, source) in parameters.items()
]

[potiuk]

Would you like to take a stab on it @wolfier ?

[ephraimbuddy]

I think we should hide this in _include_envs so it's same as _include_cmds & _include_secrets:

diff --git a/airflow/configuration.py b/airflow/configuration.py
index 41778fe374..df393da6e6 100644
--- a/airflow/configuration.py
+++ b/airflow/configuration.py
@@ -1149,7 +1149,8 @@ class AirflowConfigParser(ConfigParser):
             if not display_sensitive and env_var != self._env_var_name("core", "unit_test_mode"):
                 # Don't hide cmd/secret values here
                 if not env_var.lower().endswith("cmd") and not env_var.lower().endswith("secret"):
-                    opt = "< hidden >"
+                    if (section, key) in self.sensitive_config_values:
+                        opt = "< hidden >"
 
             elif raw:
                 opt = opt.replace("%", "%%")
(END)

[uranusjr]

I searched for as_dict usages and this is the only place where display_sensitive is False, so I think we should simply incorporate SENSITIVE_CONFIG_VALUES as Ephraim said. Otherwise the flag is basically useless.